Apparatus and Method for Computing a Matrix Vector Product of a Certain Matrix and a Vector

Simple SummaryContent extracted from patent full text and abstract with AI.

This invention describes an apparatus and method for efficiently calculating the product of a matrix and a vector. The matrix is split into vertical slices, each of which is represented by the product of multiple sparse matrices containing mostly zeros, with remaining non-zero elements being powers of two. The computation is handled by chains of processing blocks, where each block uses efficient operations—mainly shifts and additions instead of multiplications—on the input vector, making this well-suited for implementation in specialized hardware such as FPGAs or ASICs. This approach significantly reduces the computational and hardware resources required, while maintaining high accuracy, particularly benefiting neural network operations in edge and embedded environments.

Use CasesContent extracted from patent full text and abstract with AI.

• Accelerating inference in artificial neural networks (ANNs), especially in deep learning applications.
• Implementing efficient matrix-vector multiplications in resource-constrained devices like mobile phones, IoT devices, and automotive control systems.
• Energy-efficient computation for edge AI hardware, such as smart sensors or on-device machine learning.
• Hardware acceleration in reconfigurable logic devices like FPGAs, or fixed-function hardware like ASICs, in systems requiring fast and predictable computations (e.g., robotics, real-time analytics).
• Improving the performance and efficiency of recommendation engines and other large-scale data processing tasks relying on linear algebra.

BenefitsContent extracted from patent full text and abstract with AI.

• Greatly reduces hardware resource requirements (e.g., lookup tables, multipliers), leading to smaller, cheaper, and less power-hungry devices.
• Enables the use of simple shift and add operations instead of multiplications, making implementation faster and more energy-efficient.
• Meets or exceeds the accuracy of conventional implementations by optimizing matrix decomposition according to desired precision.
• Highly configurable for different matrix sizes and precision requirements, making it flexible for diverse applications.
• Facilitates deployment of advanced AI algorithms on edge devices and embedded systems where computational resources and energy budgets are limited.
• Scalable to large matrix sizes and can handle both fixed function and reconfigurable hardware architectures.

Technical Classifications (CPCs)

Inventors & Applicants

Patent Abstract

An apparatus for computing a matrix vector product of a given matrix and an arbitrary vector is described. The given matrix is represented by S submatrices, with S≥1, with each submatrix representing a vertical slice of the given matrix, and with each submatrix approximated by the product of P further matrices, with P≥1. Each further matrix is a sparse matrix and includes in each row a certain number of elements unequal to zero. The apparatus comprises S processing chains, wherein each processing chain is to receive the arbitrary vector and comprises P processing blocks. Each processing block is to multiply a block input vector and an associated further matrix by shifting the elements of the block input vector according to the values of the elements in the associated further matrix which are unequal to zero, and by combining the shifted elements of the block input vector to obtain respective elements of a block output vector.