US 12,596,918 B2 · Accelerator for deep neural networks

Abstract

Described is a system, integrated circuit and method for reducing ineffectual computations in the processing of layers in a neural network. One or more tiles perform computations where each tile receives input neurons, offsets and synapses, and where each input neuron has an associated offset. Each tile generates output neurons, and there is also an activation memory for storing neurons in communication with the tiles via a dispatcher and an encoder. The dispatcher reads neurons from the activation memory and communicates the neurons to the tiles and reads synapses from a memory and communicates the synapses to the tiles. The encoder receives the output neurons from the tiles, encodes them and communicates the output neurons to the activation memory. The offsets are processed by the tiles in order to perform computations only on non-zero neurons. Optionally, synapses may be similarly processed to skip ineffectual operations.

Claims20

Dependent claims

A method for performing computations in a neural network, the method comprising:

using encoder circuitry, identifying non-zero neurons, from among first neurons corresponding to an output of a first layer of the neural network, that have a corresponding non-zero value among respective values of the first neurons, and determining offset values of the non-zero neurons, where each offset value of the offset values has a bit-value that corresponds to a position of a corresponding non-zero neuron of the non-zero neurons, wherein the bit-values respectively reflect an index position of the corresponding non-zero neuron among indices of the first neurons;
and
using processing circuitry, generating second neurons corresponding to an output of a second layer following the first layer of the neural network by performing computations, of the second layer, using only the non-zero neurons and the offset values.

An apparatus for performing computations in a neural network, the apparatus comprising:

one or more tiles circuitries configured to perform computations, of the neural network, by processing input neurons and weights to generate output neurons of a current layer of the neural network;
an encoder circuitry configured to encode the output neurons of the current layer as input neurons of a next layer of the neural network;
and
a dispatcher circuitry configured to provide the input neurons of the next layer to the one or more tiles circuitries to process the input neurons of the next layer, wherein the encoder circuitry is configured to identify non-zero neurons from among the output neurons of the current layer of the neural network, and determine offset values of the non-zero neurons, where each offset value of the offset values has a bit-value that corresponds to a position of a corresponding non-zero neuron of the non-zero neurons;
and
wherein the one or more tile circuitries are configured to determine output neurons of the next layer by performing computations of the next layer using the non-zero neurons and the offset values.

3 more elements

Dependencies are inferred from the claim text.

Accelerator for deep neural networks

Life of this patent

Accelerator for deep neural networks

Life of this patent

Abstract

Claims20