Methods and systems for executing a neural network with bit addition-based inner product operator
OwnerXinlin LI, Vanessa COURVILLE +2inventors
InventorsXinlin LI, Vanessa COURVILLE, Vahid PARTOVI NIA, Boxing CHEN
Life of this patent
Estimating the term…
Abstract
Systems and methods for computing a neural network layer of a neural network are described. A float shift inner product operator is described for computing an inner product between a floating point input vector and a weight vector having weight values that are power of two values. An inner product is computed by performing addition on the sign bits and exponent bits of a floating point input element and corresponding weight element.
Claims20
A computing system for computing an output of a neural network layer of a neural network, the computing system comprising:
- a memory storing a weight vector for the neural network layer, each element of the weight vector being a weight element encoded as a bit string including a sign bit representing a sign value of the weight element and a set of exponent bits representing an exponent of a weight value;
- and
- a processing unit coupled to the memory, the processing unit comprising:
- circuitry configured to receive a floating point input vector to the neural network layer and the weight vector for the neural network layer, each element of the floating point input vector being a floating point input element encoded as a floating point bit string including a sign bit representing a sign value of the floating point input element and a set of exponent bits representing an exponent of the floating point input element;
- circuitry configured to compute an inner product between the floating point input vector and the weight vector by:
- for each floating point input element in the floating point input vector and a corresponding weight element in the weight vector, performing addition to add the sign bit of the weight element to the sign bit of the floating point input element and to add the set of exponent bits of the weight element to the set of exponent bits of the floating point input element, to generate a respective weighted input element;
- and
- performing floating point summation of the respective weighted input elements to generate the inner product;
- and
- circuitry configured to output the inner product as an output element of an output vector of the neural network layer.
A method for computing an output of a neural network layer of a neural network, the method comprising:
- receiving a floating point input vector, each element of the floating point input vector being a floating point input element encoded as a floating point bit string including a sign bit representing a sign value of the floating point input element and a set of exponent bits representing an exponent of the floating point input element;
- obtaining a weight vector for the neural network layer, each element of the weight vector being a weight element encoded as a bit string including a sign bit representing a sign value of the weight element and a set of exponent bits representing an exponent of a weight value;
- computing an inner product between the floating point input vector and the weight vector by:
- for each floating point input element in the floating point input vector and a corresponding weight element in the weight vector, performing addition to add the sign bit of the weight element to the sign bit of the floating point input element and to add the set of exponent bits of the weight element to the set of exponent bits of the floating point input element, to generate a respective weighted input element;
- and
- performing floating point summation of the respective weighted input elements to generate the inner product;
- and
- outputting the inner product as an output element of an output vector of the neural network layer.
Dependencies are inferred from the claim text.