US 11,775,611 B2Granted

Piecewise quantization for neural networks

AssigneeSamsung Electronics Co., Ltd.

InventorsJun Fang, Joseph H. Hassoun, Ali Shafiee Ardestani, Hamzah Ahmed Ali Abdelaziz, Georgios Georgiadis, Hui Chen, David Philip Lloyd Thorsley

Life of this patent

Estimating the term…

patentgrep

Loading patent…

Abstract

In some embodiments, a method of quantizing an artificial neural network includes dividing a quantization range for a tensor of the artificial neural network into a first region and a second region, and quantizing values of the tensor in the first region separately from values of the tensor in the second region. In some embodiments, linear or nonlinear quantization are applied to values of the tensor in the first region and the second region. In some embodiments, the method includes locating a breakpoint between the first region and the second region by substantially minimizing an expected quantization error over at least a portion of the quantization range. In some embodiments, the expected quantization error is minimized by solving analytically and/or searching numerically.

Claims20

Dependent claims

A system, comprising:

a weight buffer configured to store weight values and a first index value associated with each respective weight value, each first index value indicating a type of quantizing applied to a corresponding weight value, the weight values being arranged in at least one column of the weight buffer;
an activation buffer configured to store activation values, a second index value associated with each respective activation value, and a bit-range indicator associated with each respective activation value;
a selector circuit configured to select a weight value in each column of the weight buffer corresponding to the second index value of a current activation value;
a multiplier circuit corresponding to each respective column of the weight buffer, the multiplier circuit configured to multiply the current activation value and the weight value selected corresponding to the second index value of the current activation value;
a shift-register circuit coupled to an output of the multiplier circuit, the shift-register circuit being configured to shift the output of the multiplier circuit based on the bit-range indicator associated with the current activation value;
and
an adder-tree circuit coupled to an output of the shift-register circuit, the adder-tree circuit being configured to be enabled or disabled to add the output of the shift-register circuit based on the first index value of the weight value selected in the column of the weight buffer.

4 more elements

A system, comprising:

a first buffer configured to store first values and a first index value associated with each respective first value, each first index value indicating a type of quantizing applied to a corresponding first value, the first values being arranged in at least one column of the first buffer;
a second buffer configured to store second values, a second index value associated with each respective second value, and a bit-range indicator associated with each respective second value;
a selector circuit configured to select a first value in each column of the first buffer corresponding to the second index value of a current second value;
a multiplier circuit corresponding to each respective column of the first buffer, each multiplier circuit configured to multiply the current second value and the first value selected corresponding to the second index value of the current second value;
a shift-register circuit coupled to an output of the respective multiplier circuit, the shift-register circuit being configured to shift the output of the multiplier circuit based on the bit-range indicator associated with the current second value;
and
an adder-tree circuit coupled to an output of the shift-register circuit, the adder-tree circuit being configured to be enabled or disabled to add the output of a corresponding shift-register circuit based on the first index value of the first value selected in the column of the first buffer.

4 more elements

Dependencies are inferred from the claim text.

Facts

Application: 16/816,247Filed 11 Mar 2020 · us provisional application
Priority: 1 Nov 2019
Published: US 2021/0133278 A16 May 2021
Granted: 3 Oct 2023
Term ends: Not estimatedThe source records don’t support an estimate.

Classification9 codes

G06F 17/18
Complex mathematical operations: for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression…
G06N 3/063
Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons: using electronic means
G06N 3/02
Computing arrangements based on biological models: neural networks
G06N 3/04
Neural networks: architecture, e.g. interconnection topology