Back to results
US 11,599,334 B2Granted
Enhanced multiply accumulate device for neural networks
AssigneesVeriSilicon Microelectronics, VeriSilicon Holdings Co., Ltd.
InventorsMankit Lo, Meng Yue, Jin Zhang
Life of this patent
Estimating the term…
Abstract
A device for performing multiply/accumulate operations processes values in first and second buffers and having a first width using a computational pipeline with a second width, such as half the first width. A sequencer processes combinations of portions (high-high, low-low, high-low, low-high) of the values in the first and second buffers using a multiply/accumulate circuit and adds the accumulated result of each combination of portions to a group accumulator. Adding to the group accumulator may be preceded by left shifting the accumulated result (the first width for the high-high combination and the second width for the low-high and high-low combination).
Claims14
A device comprising:
- a first input buffer configured to store first values having a first width;
- a second input buffer configured to store second values having the first width;
- a multiply/accumulate circuit configured to perform a multiply/accumulate operation on input arguments having a second width that is half the first width;
- a group accumulator configured to accumulate an output the multiply/accumulate circuit;
- a sequencer configured to, for each combination of a plurality of combinations of portion positions that are possible for the first input buffer and the second input buffer:
- (a) for the each combination of portion positions, input portions of the first values and the second values corresponding to the each combination of portion positions to the multiplier accumulate circuit;
- and
- (c) invoke the group accumulator to add the output of the multiply/accumulate circuit to contents of a group accumulation buffer;
- wherein the portion positions for the first input buffer and the second input buffer include:
- a high portion and a low portion, the high portion having higher magnitude than the low portion, the high portion and low portion being non-overlapping and a sum of a number of bits in the high portion and a number of bits in the low portion is equal to a number of bits in the first width;
- wherein the plurality of combinations of portion positions that are possible for the first input buffer and the second input buffer include:
- the high portion of the first values in the first input buffer and the high portion of the second values in the second input buffer;
- the low portion of the first values in the first input buffer and the high portion of the second values in the second input buffer;
- the low portion of the first values in the first input buffer and the low portion of the second values in the second input buffer;
- and
- the high portion of the first values in the first input buffer and the low portion of the second values in the second input buffer;
- wherein the second width is half the first width;
- wherein the group accumulator is further configured to:
- when the portion positions for the first input buffer and the second input buffer in the each combination include the high portion for the first input buffer and the high portion for the second input buffer, shift the output of the multiply/accumulate circuit by the first width to obtain a shifted output and add the shifted output to the contents of the group accumulation buffer;
- and
- when the portion positions for the first input buffer and the second input buffer in the each combination include the high portion for only one of the first input buffer and the second input buffer, shift the output of the multiply/accumulate circuit by the second width to obtain a shifted output and add the shifted output to the contents of the group accumulation buffer.
18 more elements
A method comprising:
- providing a first input buffer configured to store first values having a first width;
- providing a second input buffer configured to store second values having the first width;
- performing a multiply/accumulate operation on high portions of the first values and high portions of the second values using a computational pipeline having a second width to obtain a first intermediate accumulation;
- increasing a group accumulation value stored in a group accumulation buffer according to the first intermediate accumulation;
- performing a multiply/accumulate operation on high portions of the first values and low portions of the second values to obtain a second intermediate accumulation;
- increasing the group accumulation value stored in the group accumulation buffer according to the second intermediate accumulation;
- performing a multiply/accumulate operation on low portions of the first values and low portions of the second values to obtain a third intermediate accumulation;
- increasing the group accumulation value stored in the group accumulation buffer according to the third intermediate accumulation;
- performing a multiply/accumulate operation on low portions of the first values and high portions of the second values to obtain a fourth intermediate accumulation;
- and
- increasing the group accumulation value stored in the group accumulation buffer according to the fourth intermediate accumulation;
- wherein the high portions of each of the first values and the second values has higher magnitude than the low portions of each of the first values and the second values, the high portions and low portions of each of the first values and the second values are non-overlapping and a sum of a number of bits in the high portions and a number of bits in the low portions is equal to a number of bits in the first width;
- and
- wherein increasing the group accumulation value stored in the group accumulation buffer according to the first intermediate accumulation comprises:
- increasing the group accumulation value stored in the group accumulation buffer by the first intermediate accumulation left shifted by the first width.
12 more elements
Dependencies are inferred from the claim text.