US 12,688,254 B1Granted

Method and apparatus for logarithmic mathematics in AI matrix multiplications

AssigneeTensordyne, Inc.

InventorsJian Hui Huang, Gary S. Goldman, Jan Lennart Haug, Michael Wilhelm Laraia, Lukas Rinder

Life of this patent

Estimating the term…

Loading patent…

Abstract

A matrix multiplication system and method are disclosed that perform multiplication in a logarithmic domain and accumulation in a linear domain using co-designed bidirectional conversion circuits. Linear-domain operands are converted to a logarithmic domain by a linear-to-logarithmic conversion that divides a mantissa into exactly four non-uniform windows and applies affine mappings implemented using shift and add operations. Logarithmic-domain multiplication results are converted to linear-domain values by a logarithmic-to-linear conversion that divides a mantissa into exactly four equal-width windows and applies corresponding affine mappings implemented using shift and add operations. The affine mappings of the linear-to-logarithmic conversion are analytic inverses of the affine mappings of the logarithmic-to-linear conversion, and window boundaries of the linear-to-logarithmic conversion are derived by inverse mapping of the logarithmic-to-linear window boundaries. The conversion circuits are co-designed to provide round-trip exactness apart from quantization effects, thereby avoiding systematic bias during accumulation.

Claims30

Dependent claims

A processing system for performing matrix multiplication, comprising:

a matrix multiplication engine comprising a plurality of logarithmic logic cores, each logarithmic logic core configured to perform multiplication in a logarithmic domain and accumulation in a linear domain;
a linear-to-logarithmic conversion circuit positioned upstream of the matrix multiplication engine and shared by the plurality of logarithmic logic cores, the linear-to-logarithmic conversion circuit configured to convert input operand values from the linear domain to the logarithmic domain prior to distribution to the logarithmic logic cores, the linear-to-logarithmic conversion circuit comprising:
a mantissa window selector configured to divide a mantissa range of a linear-domain operand into exactly four non-uniform windows;
and
for each of the four non-uniform windows, a corresponding affine mapping implemented using only shift and add operations to generate a logarithmic-domain mantissa value;
a logarithmic-to-linear conversion circuit disposed within each logarithmic logic core and configured to convert logarithmic-domain multiplication results to linear-domain values prior to accumulation, the logarithmic-to-linear conversion circuit comprising:
a mantissa window selector configured to divide a mantissa range [0,1) of a logarithmic-domain value into exactly four equal-width windows, each identifiable based on most-significant mantissa bits;
and
for each of the four equal-width windows, a corresponding affine mapping implemented using only shift and add operations to generate a linear-domain mantissa value;
wherein:
each affine mapping of the linear-to-logarithmic conversion circuit is an analytic inverse of a corresponding affine mapping of the logarithmic-to-linear conversion circuit;
non-uniform window boundaries of the linear-to-logarithmic conversion circuit are computed from window boundaries of the logarithmic-to-linear conversion circuit by inverse mapping;
and
the linear-to-logarithmic conversion circuit and the logarithmic-to-linear conversion circuit are co-designed such that a round-trip conversion of a mantissa through both conversion circuits is mathematically exact apart from quantization effects, thereby avoiding systematic bias during accumulation within the matrix multiplication engine.

11 more elements

A method for performing matrix multiplication by a processing system comprising a matrix multiplication engine comprising a plurality of logarithmic logic cores, a linear-to-logarithmic conversion circuit and a logarithmic-to-linear conversion circuit, each logarithmic logic core configured to perform multiplication in a logarithmic domain and accumulation in a linear domain, the method comprising:

converting, by the linear-to-logarithmic conversion circuit comprising a mantissa window selector, wherein the linear-to-logarithmic conversion circuit is positioned upstream of the matrix multiplication engine and shared by the plurality of logarithmic logic cores, input operand values from a linear domain to a logarithmic domain using a linear-to-logarithmic conversion, the linear-to-logarithmic conversion comprising:
dividing, by the mantissa window selector, a mantissa range of each linear-domain operand into exactly four non-uniform windows;
and
applying, within each of the four non-uniform windows, a corresponding affine mapping implemented using only shift and add operations to generate a logarithmic-domain mantissa value;
distributing the operand values in logarithmic domain to the plurality of logarithmic logic cores;
within each of the plurality of logarithmic logic cores:
performing multiplication in the logarithmic domain to produce logarithmic-domain multiplication results;
converting, by the logarithmic-to-linear conversion circuit disposed within each logarithmic logic core, the logarithmic-domain multiplication results to linear-domain values using a logarithmic-to-linear conversion, the logarithmic-to-linear conversion comprising:
dividing a mantissa range [0,1) of each logarithmic-domain multiplication result into exactly four equal-width windows;
and
applying, within each equal-width window, a corresponding affine mapping implemented using only shift and add operations to generate a linear-domain mantissa value;
and
accumulating the linear-domain values;
wherein:
affine mappings used in the linear-to-logarithmic conversion are analytic inverses of corresponding affine mappings used in the logarithmic-to-linear conversion;
boundaries of the four non-uniform windows used in the linear-to-logarithmic conversion are computed from window boundaries of the logarithmic-to-linear conversion by inverse mapping;
and
a round-trip conversion of a mantissa from the linear domain to the logarithmic domain and back to the linear domain is mathematically exact apart from quantization effects, thereby avoiding systematic bias during accumulation.

15 more elements

Dependencies are inferred from the claim text.