US 2019/0378001 A1 · NEURAL NETWORK HARDWARE ACCELERATION WITH STOCHASTIC ADAPTIVE RESOURCE ALLOCATION

Abstract

A digital circuit for accelerating computations of an artificial neural network model includes a pairs selection unit that selects different subsets of pairs of input vector values and corresponding weight vector values to be processed simultaneously at each time step; a sorting unit that simultaneously processes a vector of input-weight pairs wherein pair values whose estimated product is small are routed with a high probability to small multipliers, and pair values whose estimated product is greater are routed with a high probability to large multipliers that support larger input and output values; and a core unit that includes a plurality of multiplier units and a plurality of adder units that accumulate output results of the plurality of multiplier units into one or more output values that are stored back into the memory, where the plurality of multiplier units include the small multipliers and the large multipliers.

Claims20

Dependent claims

A digital circuit for accelerating computations of an artificial neural network model, comprising:

a memory that stores an input vector, a weight vector, and an output value;
a pairs selection unit that selects different subsets of pairs of input vector values and corresponding weight vector values from the memory, to be processed simultaneously at each time step;
a sorting unit that simultaneously processes a vector of input-weight pairs wherein pair values whose estimated product is small are routed with a high probability to small multipliers, and pair values whose estimated product is greater are routed with a high probability to large multipliers that support larger input and output values;
and
a core unit that includes a plurality of multiplier units and a plurality of adder units that accumulate output results of the plurality of multiplier units into one or more output values that are stored back into the memory, wherein the plurality of multiplier units include the small multipliers and the large multipliers.

2 more elements

A method of processing a layer in an artificial neural network model, comprising the steps of:

receiving a plurality of pairs of input vector component values and corresponding weight vector component values and forming a vector of activation-weight pairs of an input vector component value v and a corresponding weight vector component value w;
calculating, for each activation-weight pair, a value of a function g(v,w) of v and w;
partially sorting the activation-weight pairs vector by repeatedly comparing pairs of function values g(v,w), and conditionally swapping corresponding activation-weight pairs in the vector based on a comparison result, wherein a number of comparisons is less than or equal to a number of all possible activation-weight pair combinations, wherein activation-weight pairs (v,w) with a smaller function value g(v,w) among the function values g(v,w) are moved with high probability to a first side of the vector, while activation-weight pairs (v,w) with a large function value g(v,w) among the function values g(v,w) are moved with high probability to a second side of the vector opposite form the first side;
routing the partially sorted activation-weight pairs (v,w) based on the comparison result wherein activation-weight pairs (v,w) from the first side of the vector are routed to small multipliers, and activation-weight pairs (v,w) from the second side of the vector are routed to a large multiplier;
and
processing the activation pairs routed to the multipliers by multiplying the pairs of values and then accumulating the product results of the multiplications to obtain an output vector value for a current layer, wherein those activation-weight pairs whose function value is small are with a high probability multiplied by a small multiplier to yield a small product, and those activation-weight pairs whose function value is large are with a high probability multiplied by a large multiplier to yield a large product

3 more elements

Dependencies are inferred from the claim text.

Neural network hardware acceleration with stochastic adaptive resource allocation

Life of this patent

Neural network hardware acceleration with stochastic adaptive resource allocation

Life of this patent

Abstract

Claims20