Neural network unit employing user-supplied reciprocal for normalizing an accumulated value
AssigneeVia Alliance Semiconductor Co., Ltd.
InventorsG. Glenn Henry, Terry Parks
Life of this patent
Estimating the term…
Abstract
A neural network unit including a register programmable with a representation of a reciprocal value of a divisor and a plurality of neural processing units (NPU). Each NPU has an ALU, an accumulator, and a reciprocal multiplier unit. The ALU performs arithmetic and logical operations on a sequence of operands to generate a sequence of results and accumulates the sequence of results as an accumulated value into the accumulator. The reciprocal multiplier unit receives the representation of the reciprocal value and the accumulated value and in response generates a result that is the quotient of the accumulated value and the divisor.
Claims14
A neural network unit at least partially implemented in hardware, performing a convolution operation or a multiply-accumulate and activation function operation in a hybrid fashion, comprising:
- a register programmable with a representation of a reciprocal value of a divisor and an activation function field;
- a first memory;
- a second memory;
- and
- a plurality of neural processing units (NPU), each comprising:
- an arithmetic logic unit (ALU) that performs arithmetic and logical operations on a sequence of operands to generate a sequence of results;
- an accumulator into which the ALU accumulates the sequence of results as an accumulated value;
- a plurality of activation function units, each applying an activation function on the accumulated value to generate an activation function result, wherein the activation function field specifies the activation function applied to the accumulated value;
- and
- a reciprocal multiplier unit that receives the representation of the reciprocal value and the accumulated value and in response generates a reciprocal multiplication result that is the quotient of the accumulated value and the divisor;
- wherein when the neural network unit performs the convolution operation, the first memory holds elements of a data matrix and the second memory holds elements of a convolution kernel, and the accumulated value is a convolution of the convolution kernel and a respective sub-matrix of the elements of the data matrix;
- wherein when the neural network unit performs the multiply-accumulate and activation function operation, the first memory holds a plurality of data words and the second memory holds a plurality of weight words, and the accumulated value is a sum of products generated by multiplying each of the plurality of data words by each of the corresponding weight words.
A method for operating a neural network unit performing a convolution operation or a multiply-accumulate and activation function operation in a hybrid fashion and having a plurality of neural processing units (NPU) each having an arithmetic logic unit (ALU), an accumulator, and a reciprocal multiplier unit, the method comprising:
- programming a register with a representation of a reciprocal value of a divisor and an activation function field;
- performing, by each of the ALUs, arithmetic and logical operations on a sequence of operands to generate a sequence of results;
- accumulating, by each of the ALUs, into the an accumulator the sequence of results as an accumulated value;
- applying, by each of a plurality of activation function units, an activation function on the accumulated value to generate an activation function result, wherein the activation function field specifies the activation function applied to the accumulated value;
- receiving, by the reciprocal multiplier unit, the representation of the reciprocal value and the accumulated value and in response generating a reciprocal multiplication result that is the quotient of the accumulated value and the divisor;
- and
- when the neural network unit performs the convolution operation, holding, by a first memory, elements of a data matrix, and holding, by a second memory, elements of a convolution kernel, wherein the accumulated value is a convolution of the convolution kernel and a respective sub-matrix of the elements of the data matrix;
- wherein when the neural network unit performs the multiply-accumulate and activation function operation, the first memory holds a plurality of data words and the second memory holds a plurality of weight words, and the accumulated value is a sum of products generated by multiplying each of the plurality of data words by each of the corresponding weight words.
A computer program product encoded in at least one non-transitory computer usable medium for use with a computing device, the computer program product comprising:
- computer usable program code embodied in said medium, for specifying a neural network unit performing a convolution operation or a multiply-accumulate and activation function operation in a hybrid fashion, the computer usable program code comprising:
- first program code for specifying a first memory, a second memory, and a register programmable with a representation of a reciprocal value of a divisor and activation function field;
- and
- second program code for specifying a plurality of neural processing units (NPU), each comprising:
- an arithmetic logic unit (ALU) that performs arithmetic and logical operations on a sequence of operands to generate a sequence of results;
- an accumulator into which the ALU accumulates the sequence of results as an accumulated value;
- a plurality of activation function units, each applying an activation function on the accumulated value to generate an activation function result, wherein the activation function field specifies the activation function applied to the accumulated value;
- and
- a reciprocal multiplier unit that receives the representation of the reciprocal value and the accumulated value and in response generates a reciprocal multiplication result that is the quotient of the accumulated value and the divisor;
- wherein when the neural network unit performs the convolution operation, the first memory holds elements of a data matrix, the second memory holds elements of a convolution kernel, and the accumulated value is a convolution of a convolution kernel and a respective sub-matrix of elements of a data matrix;
- wherein when the neural network unit performs the multiply-accumulate and activation function operation, the first memory holds a plurality of data words and the second memory holds a plurality of weight words, and the accumulated value is a sum of products generated by multiplying each of the plurality of data words by each of the corresponding weight words.
Dependencies are inferred from the claim text.