Systems and methods for energy-efficient analog matrix multiplication for machine learning processes
AssigneeAnalog Devices, Inc.
InventorsSung Ung Kwak, Robert Michael Muchsel
Life of this patent
Estimating the term…
Abstract
An energy-efficient multiplication circuit uses analog multipliers and adders to reduce the distance that data has to move and the number of times that the data has to be moved when performing matrix multiplications in the analog domain. The multiplication circuit is tailored to bitwise multiply the innermost product of a rearranged matrix formula generate a matrix multiplication result in form of a current that is then digitized for further processing.
Claims20
A matrix multiplication system comprising:
- a sub-circuit comprising:
- a magnitude adjustment circuit that adjusts magnitudes of values, which have been obtained from a data memory, to generate magnitude-adjusted values;
- and
- a multiplier that multiples the magnitude-adjusted values with a column weight to obtain a partial product;
- an adder that adds outputs of one or more multipliers to obtain a sum of partial products;
- and
- an activation that applies an activation function to the sum of partial products to generate a matrix multiplication result.
A method for performing an analog matrix multiplication, the method comprising:
- using an analog multiplication circuit to multiply bits of data items with weights to obtain products that correspond to sets of currents;
- adding two or more currents in the sets of currents to generate a sum of partial dot products in a matrix multiplication;
- weighting an output current, which is associated with the sum, according to a bit position to generate a result;
- and
- applying an activation function to the result to generate a matrix multiplication result.
An analog multiplication circuit comprising:
- a set of switches that represents at least one of a set of weight bits and a set of data bits for currents flowing through the set of switches, the analog multiplication circuit performing steps comprising:
- multiplying data bits from the set of data bits with weight bits from the set of weight bits to obtain products that correspond to a set of currents;
- and
- adding at least some of the currents flowing through the set of switches to obtain an output current that represents a sum of partial dot products in a matrix-multiplication.
Dependencies are inferred from the claim text.