Digital neural network
AssigneeSicily Merger Sub II, Inc.
InventorsPhilip Winterbottom, Martinus Bos
Life of this patent
Estimating the term…
Abstract
Various embodiments provide for digital neural network (DNN) that form part of a machine learning (ML) processor and that perform a compute intensive function, such as convolutions or matrix multiplies, which can facilitate operation of a ML model. According to some embodiments, the DNN comprises a combinatorial tree that uses multiple-accumulate (MAC) units, and a sequencer that reads values from memory devices and feeds the combinatorial tree.
Claims23
A hardware circuit comprising:a combinatorial tree having a plurality of rows and a plurality of columns, the combinatorial tree using a plurality of hardware units to generate one or more dot product values based on input values received at a first column the combinatorial tree;a first memory device for storing the input values;a sequencer configured to obtain the input values from the first memory device and reorganize the input values into reorganized input values in a switch fabric;a clock configured to control a time for when a portion of the reorganized input values is provided to the first column of the combinatorial tree from the switch fabric;
- anda plurality of hardware units where each hardware unit includes an accumulator configured to accumulate the one or more dot product values,wherein during one or more subsequent time periods defined by the clock, additional portions of the reorganized input values are provided to accumulators of hardware units associated with the first column, andwherein during one or more additional time periods controlled by the clock, a value associated with each of the accumulators of the hardware units in the first column is written to the first memory device.
A method comprising:reading, by a sequencer, a first plurality of weight values and a first plurality of operand values from a set of memory devices operatively coupled to the sequencer;receiving, at a first column of a combinatorial tree, a plurality of first input vectors from the sequencer, the first plurality of input vectors including the first plurality of weight values and the first plurality of operand values;processing the first plurality of input vectors by math units at the first column of the combinatorial tree, the math units being associated with each of the input vectors, each of the math units having at least one accumulator, the combinatorial tree being configured to generate a dot product value in an accumulator of each of the math units based on values received by the math units at the first column of the combinatorial tree;reading, by the sequencer, a second plurality of weight values and a second plurality of operand values from the set of memory devices;
- receiving, at the first column of the combinatorial tree, a second plurality of input vectors from the sequencer, the second plurality of input vectors including the second plurality of weight values and the second plurality of operand values;
- processing the second plurality of input vectors by the math units at the first column of the combinatorial tree, the first plurality of input vectors and the second plurality of inputs vector being part of a sequence of input vectors to be received by the math units in the first column of the combinatorial tree;
- and
- after all of the sequence of input vectors has been received by the first column of the combinatorial tree and processed by associated math units, providing a set of final dot product values by retrieving a value from each accumulator in each of the math units residing in every row of the first column of the combinatorial tree.
A hardware circuit comprising:an array of dot product units organized as a combinatorial tree that generates dot product values based on weight values and activation values received as a values input by the array of dot product units, the array of dot product units includes 64 rows by 32 columns of dot product units, each dot product unit configured to receive as input two 8-bit values and having an accumulator;
- anda switch fabric operatively coupling the array to a set of memory devices, the set of memory devices storing a plurality of weight values and a plurality of activation values, the switch fabric configured to provide 64 input vectors as a vector input to a first column of the array, during a first clock sequence comprising one or more clock cycles,wherein one or more additional clock sequences cause the switch fabric to provide a plurality of additional sets of 64 input vectors to the first column of the array, and
- wherein when each of the accumulators in the first column of the array has 32 bits, writing each value from each accumulator back into the set of memory devices.
Dependencies are inferred from the claim text.