System and method of transposed matrix-vector multiplication

A hardware accelerator, comprising:

• at least one processor;
• and
• at least one compute tile including and coupled with the at least one processor, the at least one compute tile including a memory and hardware compute logic, the memory including a plurality of memory cells storing a plurality of weights corresponding to a matrix, the hardware compute logic being coupled with the plurality of memory cells and configured to perform a vector-matrix multiplication (VMM) for the matrix and for a matrix transpose for the plurality of weights being stationary for the plurality of memory cells, wherein the hardware compute logic includes:
• multiplication circuitry for the plurality of memory cells, the multiplication circuitry multiplying at least a portion of a weight stored in a memory cell with a portion of an input vector to provide a product corresponding to the memory cell;
• selection logic;
• and
• a plurality of adder trees coupled with the selection logic and the memory, the plurality of adder trees for accumulating a sum of the product of each memory cell for a portion of the plurality of memory cells, the selection logic being configured to select a first portion of the plurality of adder trees for the VMM of the matrix and to select a second portion of the plurality of adder trees for the VMM of the matrix transpose, wherein the product is selectively provided to at least one of the first portion of the plurality of adder trees for the VMM of the matrix or to the second portion of the plurality of adder trees for the VMM of the matrix transpose,
• wherein the selection logic further includes a plurality of input multiplexers, the plurality of input multiplexers being configured to route an element of the input vector to a corresponding multiplication circuitry based on whether the VMM is for the matrix or the matrix transpose.

A method, comprising:

• providing an input vector to a plurality of compute engines coupled with a processor, the plurality of compute engines including a compute-in-memory (CIM) hardware module, the CIM hardware module including a memory and hardware compute logic, the memory including a plurality of memory cells storing a plurality of weights corresponding to a matrix, the hardware compute logic coupled with the plurality of memory cells and configured to perform a vector-matrix multiplication (VMM) for the matrix and for a matrix transpose for the plurality of weights being stationary for the plurality of memory cells, the hardware compute logic including:
• multiplication circuitry for the plurality of memory cells, the multiplication circuitry multiplying at least a portion of a weight stored in a memory cell with a portion of the input vector to provide a product corresponding to the memory cell;
• selection logic;
• and
• a plurality of adder trees coupled with the selection logic and the memory, the plurality of adder trees for accumulating a sum of the product of each memory cell for a portion of the plurality of memory cells, the selection logic being configured to select a first portion of the plurality of adder trees for the VMM of the matrix and to select a second portion of the plurality of adder trees for the VMM of the matrix transpose, wherein the product is selectively provided to at least one of the first portion of the plurality of adder trees for the VMM of the matrix or to the second portion of the plurality of adder trees for the VMM of the matrix transpose,
• selecting the matrix or the matrix transpose;
• and
• performing the VMM of the input vector and the matrix or the matrix transpose based on the selecting and using at least one of the plurality of compute engines,
• wherein the selection logic further includes a plurality of input multiplexers, the plurality of input multiplexers being configured to route an element of the input vector to a corresponding multiplication circuitry based on whether the VMM is for the matrix or the matrix transpose.