Back to results
US 11,442,786 B2Granted
Computation method and product thereof
AssigneeShanghai Cambricon Information Technology Co., Ltd.
InventorsShaoli Liu, Xishan Zhang
Life of this patent
Estimating the term…
Abstract
The present disclosure provides a computation method and product thereof. The computation method adopts a fusion method to perform machine learning computations. Technical effects of the present disclosure include fewer computations and less power consumption.
Claims14
A neural network operation module configured to perform operations of a multi-layer neural network, comprising:
- a storage unit configured to store input neuron precision, weight precision, and output neuron gradient precision;
- a controller unit configured to obtain input neuron precision Sx(l), weight precision Sw(l), and output neuron gradient precision S∇x(l) of an Lth layer of the multi-layer neural network, wherein L is an integer greater than 0;
- obtain gradient update precision T according to the input neuron precision Sx(l), the weight precision Sw(l), and the output neuron gradient precision S∇x(l);
- and if the gradient update precision T is greater than preset precision Tr, adjust the input neuron precision Sx(l), the weight precision Sw(l), and the output neuron gradient precision S∇x(l) to minimize an absolute value of a difference between the gradient update precision T and the preset precision Tr;
- wherein the controller unit is further configured to obtain the preset precision Tr according to a method of machine learning;
- and
- an operating unit configured to represent an output neuron and a weight of the Lth layer according to the adjusted input neuron precision Sx(l) and the adjusted weight precision Sw(l) and represent an output neuron gradient of the Lth layer obtained from computations according to the adjusted output neuron gradient precision S∇x(l) for subsequent computations;
- wherein obtaining, by the controller unit, gradient update precision T according to the input neuron precision Sx(l), the weight precision Sw(l), and the output neuron gradient precision S∇x(l) includes:
- the controller unit performs computations on the input neuron precision Sx(l), the weight precision Sw(l), and the output neuron gradient precision S∇x(l) according to a preset formula to obtain the gradient update precision T, wherein
- the preset formula is:
- T=Sx(l)+S∇x(l)−Sw(l).
8 more elements
A neural network operation method, comprising:
- obtaining input neuron precision Sx(l), weight precision Sw(l), and output neuron gradient precision S∇x(l) of an Lth layer of a neural network;
- obtaining gradient update precision T by performing computations according to the input neuron precision Sx(l), the weight precision Sw(l), and the output neuron gradient precision S∇x(l);
- if the gradient update precision T is greater than preset precision Tr, adjusting the input neuron precision Sx(l), the weight precision Sw(l), and the output neuron gradient precision S∇x(l) to minimize an absolute value of a difference between the gradient update precision T and the preset precision Tr;
- obtaining the preset precision Tr according to a method of machine learning;
- and
- representing an output neuron and a weight of the Lth layer according to the adjusted input neuron precision Sx(l) and the weight precision Sw(l);
- and representing an output neuron gradient of the Lth layer obtained from computations according to the adjusted output neuron gradient precision S∇x(l) for subsequent computations;
- wherein obtaining the gradient update precision T by performing computations according to the input neuron precision Sx(l), the weight precision Sw(l), and the output neuron gradient precision S∇x(l) includes:
- performing computations on the input neuron precision Sx(l), the weight precision Sw(l), and the output neuron gradient precision S∇x(l) according to a preset formula to obtain the gradient update precision T,
- wherein the preset formula is:
- T=Sx(l)+S∇x(l)−Sw(l).
8 more elements
Dependencies are inferred from the claim text.