Back to results
US 11,442,785 B2Granted
Computation method and product thereof
AssigneeShanghai Cambricon Information Technology Co., Ltd.
InventorsShaoli Liu, Xishan Zhang
Life of this patent
Estimating the term…
Abstract
The present disclosure provides a computation method and product thereof. The computation method adopts a fusion method to perform machine learning computations. Technical effects of the present disclosure include fewer computations and less power consumption.
Claims14
A neural network operation module configured to perform operations of a multi-layer neural network, comprising:
- a storage unit configured to store input neuron precision, weight precision, and output neuron gradient precision;
- a controller unit configured to obtain input neuron precision Sx(l), weight precision Sw(l), and output neuron gradient precision S∇x(l) of an Lth layer of the multi-layer neural network, wherein L is an integer greater than 0, obtain gradient update precision T according to the input neuron precision Sx(l), the weight precision Sw(l), and the output neuron gradient precision S∇x(l), and if the gradient update precision T is less than preset precision Tr, adjust the input neuron precision Sx(l), the weight precision Sw(l), and the output neuron gradient precision S∇x(l) to minimize an absolute value of a difference between the gradient update precision T and the preset precision Tr;
- and
- an operating unit configured to represent an input neuron and a weight of the Lth layer according to the adjusted input neuron precision Sx(l) and the weight precision Sw(l), and represent an output neuron gradient of the Lth layer obtained from computations according to the adjusted output neuron gradient precision S∇x(l) for subsequent computations;
- wherein the controller unit is further configured to:
- obtain the preset precision Tr according to a method of machine learning, or
- obtain the preset precision Tr according to a count of output neurons, a learning rate, and a count of samples during batch processing of an (L−1)th layer, wherein the greater the count of output neurons, the count of samples during batch processing, and the learning rate of the (L−1)th layer are, the greater the preset precision Tr is.
4 more elements
A neural network operation method, comprising:
- obtaining input neuron precision Sx(l), weight precision Sw(l), and output neuron gradient precision S∇x(l) of an Lth layer of a neural network;
- obtaining gradient update precision T by performing computations according to the input neuron precision Sx(l), the weight precision Sw(l), and the output neuron gradient precision S∇x(l);
- if the gradient update precision T is less than preset precision Tr, adjusting the input neuron precision Sx(l), the weight precision Sw(l), and the output neuron gradient S∇x(l) to minimize an absolute value of a difference between the gradient update precision T and the preset precision Tr;
- representing an input neuron and a weight of the Lth layer according to the adjusted input neuron precision Sx(l) and the weight precision Sw(l);
- and
- representing an output neuron gradient of the Lth layer obtained from computations according to the adjusted output neuron gradient precision S∇x(l) for subsequent computations,
- wherein the method further comprises:
- obtaining the preset precision Tr according to a method of machine learning, or
- obtaining the preset precision Tr according to a count of output neurons, a learning rate, and a count of samples during batch processing of an (L−1)th layer, wherein the greater the count of output neurons, the count of samples during batch processing, and the learning rate of the (L−1)th layer are, the greater the preset precision Tr is.
6 more elements
Dependencies are inferred from the claim text.