US 2026/0147536 A1 · ALIGNMENT IN HARDWARE ACCELERATORS

Abstract

Systems, apparatuses, and methods are disclosed for improved matrix-vector operations in accelerators that may be useful or heavy AI training and inference workloads. The disclosed technology provides arrangements that permit more efficient computation by, for example, performing calculations without repeated conversion between numeric domains. In some implementations, a compute-in-memory (CIM) macro is configured to perform a vector matrix multiplication (VMM) operation between a vector of activation values and a matrix of weight values in floating point formats. The CIM macro has a functional block configured to align mantissa bits of primitive products between the activation values and the weight values by shifting the mantissa bits and an adder tree configured to output an accumulation value of the primitive products in an integer format by adding the shifted mantissa bits.

Claims21

Dependent claims

A computing system comprising:

a compute-in-memory (CIM) macro comprising columns of computing units, the columns of computing units comprising:
a column cell configured to generate a primitive product of an activation value and a weight value in a floating point format;
a functional block configured to align mantissa bits of primitive products generated by column cells in the computing unit by shifting the mantissa bits;
and
an adder tree configured to output an accumulation value of the primitive products in an integer format by adding the shifted mantissa bits.

2 more elements

Dependencies are inferred from the claim text.

Alignment in hardware accelerators

Life of this patent

Alignment in hardware accelerators

Life of this patent

Abstract

Claims21