# US 10,373,050 B2 · Fixed point neural network based on floating point neural network quantization

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,373,050 B2
- Document ID: US10373050B2
- Status: Granted
- Kind: B2
- Application number: 14/920,099
- Filed: 2015-10-22
- Published: 2016-11-10
- Granted: 2019-08-06
- Earliest claimed: 2015-05-08
- Assignees: QUALCOMM Incorporated
- Applicants: QUALCOMM Incorporated
- Inventors: Dexu Lin; Venkata Sreekanta Reddy Annapureddy; David Edward Howard; David Jonathan Julian; Somdeb Majumdar; William Richard Bell, II
- Web page: https://patentgrep.com/patent/US10373050B2

## CPC

- G06N 3/08: Learning methods
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/048: Activation functions
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- G06N 3/10: Interfaces, programming languages or software development kits, e.g. for simulating neural networks
- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06V 10/82: using neural networks
- G06N 3/06: Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N 3/063: using electronic means
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method of quantizing a floating point machine learning network to obtain a fixed point machine learning network using a quantizer may include selecting at least one moment of an input distribution of the floating point machine learning network. The method may also include determining quantizer parameters for quantizing values of the floating point machine learning network based at least in part on the at least one selected moment of the input distribution of the floating point machine learning network to obtain corresponding values of the fixed point machine learning network.

## Claims

### Claim 1

A method of quantizing a floating point machine learning network to obtain a fixed point machine learning network using a quantizer, comprising:
selecting a moment of an input distribution of the floating point machine learning network, the moment comprising at least one of a mean or a variance;
shifting activation values to create a zero-mean distribution for each layer of the fixed point machine learning network;
shifting a coordinate of any non-linear function applied to a shifted activation value by an amount corresponding to the shifted activation value; and
quantizing values of the floating point machine learning network based at least in part on quantizer parameters determined from the selected moment of the input distribution of the floating point machine learning network to obtain corresponding values of the fixed point machine learning network.

### Claim 2 (depends on 1)

The method of claim 1, in which the quantizer parameters for quantizing values of the floating point machine learning network comprise a maximum dynamic encoding range of the fixed point machine learning network.

### Claim 3 (depends on 1)

The method of claim 1, in which the quantizer parameters for quantizing values of the floating point machine learning network comprise a quantizer step size.

### Claim 4 (depends on 3)

The method of claim 3, further comprising determining the quantizer step size of the quantizer for bias, weight, and/or activation values based at least in part on a mean value and standard deviation of the bias, the weight, and/or the activation values from the floating point machine learning network.

### Claim 5 (depends on 1)

The method of claim 1, in which the quantizer parameters for quantizing values of the floating point machine learning network comprise a bit width.

### Claim 6 (depends on 1)

The method of claim 1, in which the quantizer parameters for quantizing values of the floating point machine learning network comprise a signed or unsigned indicator value.

### Claim 7 (depends on 1)

The method of claim 1, in which the corresponding values of the fixed point machine learning network comprise a bias, a weight, and/or activation values.

### Claim 8

A method of converting a first machine learning network to a second machine learning network, comprising:
obtaining a mean value of a distribution of activation values of the first machine learning network;
applying the mean value to bias values of the first machine learning network to obtain modified bias values for the second machine learning network, such that activation values for each layer of the second machine learning network have a zero mean distribution; and
quantizing the activation values stored in a memory based on quantizer parameters.

### Claim 9 (depends on 8)

The method of claim 8, further comprising modifying a non-linear activation function when the mean value is incorporated into the bias values.

### Claim 10 (depends on 8)

The method of claim 8, in which the first machine learning network comprises a floating point neural network and the second machine learning network comprises a fixed point neural network.

### Claim 11

An apparatus for quantizing a floating point machine learning network to obtain a fixed point machine learning network using a quantizer, comprising:
means for selecting a moment of an input distribution of the floating point machine learning network, the moment comprising at least one of a mean or a variance;
means for shifting activation values to create a zero-mean distribution for each layer of the fixed point machine learning network;
means for shifting a coordinate of any non-linear function applied to a shifted activation value by an amount corresponding to the shifted activation value; and
means for quantizing values of the floating point machine learning network based at least in part on quantizer parameters determined from the selected moment of the input distribution of the floating point machine learning network to obtain corresponding values of the fixed point machine learning network.

### Claim 12 (depends on 11)

The apparatus of claim 11, in which the quantizer parameters for quantizing values of the floating point machine learning network comprise a maximum dynamic encoding range of the fixed point machine learning network, a bit width, a signed or unsigned indicator value and/or a quantizer step size.

### Claim 13 (depends on 12)

The apparatus of claim 12, further comprising means for determining the quantizer step size of the quantizer for bias, weight, and/or activation values based at least in part on a mean value and standard deviation of the bias, the weight, and/or the activation values from the floating point machine learning network.

### Claim 14 (depends on 11)

The apparatus of claim 11, in which the corresponding values of the fixed point machine learning network comprise a bias, a weight, and/or activation values.

### Claim 15

An apparatus for quantizing a floating point machine learning network to obtain a fixed point machine learning network using a quantizer, comprising:
a memory unit; and
at least one processor coupled to the memory unit, the at least one processor configured:
to select a moment of an input distribution of the floating point machine learning network, the moment comprising at least one of a mean or a variance;
to shift activation values to create a zero-mean distribution for each layer of the fixed point machine learning network;
to shift a coordinate of any non-linear function applied to a shifted activation value by an amount corresponding to the shifted activation value; and
to quantize values of the floating point machine learning network based at least in part on quantizer parameters determined from the selected moment of the input distribution of the floating point machine learning network to obtain corresponding values of the fixed point machine learning network.

### Claim 16 (depends on 15)

The apparatus of claim 15, in which the quantizer parameters for quantizing values of the floating point machine learning network comprise a maximum dynamic encoding range of the fixed point machine learning network, a bit width, a signed or unsigned indicator value and/or a quantizer step size.

### Claim 17 (depends on 16)

The apparatus of claim 16, in which the at least one processor is further configured to determine the quantizer step size of the quantizer for bias, weight, and/or activation values based at least in part on a mean value and standard deviation of the bias, the weight, and/or the activation values from the floating point machine learning network.

### Claim 18 (depends on 15)

The apparatus of claim 15, in which the corresponding values of the fixed point machine learning network comprise a bias, a weight, and/or activation values.

### Claim 19

A non-transitory computer-readable medium having program code recorded thereon for quantizing a floating point machine learning network to obtain a fixed point machine learning network using a quantizer, the program code being executed by a processor and comprising:
program code to select a moment of an input distribution of the floating point machine learning network, the moment comprising at least one of a mean or a variance;
program code to shift activation values to create a zero-mean distribution for each layer of the fixed point machine learning network;
program code to shift a coordinate of any non-linear function applied to a shifted activation value by an amount corresponding to the shifted activation value; and
program code to quantize values of the floating point machine learning network based at least in part on quantizer parameters determined from the moment of the input distribution of the floating point machine learning network to obtain corresponding values of the fixed point machine learning network.

### Claim 20 (depends on 19)

The computer-readable medium of claim 19, in which the quantizer parameters for quantizing values of the floating point machine learning network comprise a maximum dynamic encoding range of the fixed point machine learning network, a bit width, a signed or unsigned indicator value and/or a quantizer step size.

### Claim 21 (depends on 20)

The computer-readable medium of claim 20, further comprising program code to determine the quantizer step size of the quantizer for bias, weight, and/or activation values based at least in part on a mean value and standard deviation of the bias, the weight, and/or the activation values from the floating point machine learning network.

### Claim 22 (depends on 19)

The computer-readable medium of claim 19, in which the corresponding values of the fixed point machine learning network comprise a bias, a weight, and/or activation values.
