# US 2023/0410255 A1 · Decreased quantization latency

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0410255 A1
- Document ID: US20230410255A1
- Status: Published application
- Kind: A1
- Application number: 18/251,220
- Filed: 2021-01-22
- Published: 2023-12-21
- Earliest claimed: 2021-01-22
- Assignees: QUALCOMM Incorporated
- Applicants: QUALCOMM Incorporated
- Inventors: Wenhao ZHANG; Zhiguo LI; Ronghui LIN; Zhiping PANG
- Web page: https://patentgrep.com/patent/US20230410255A1

## CPC

- G06T 3/4046: using neural networks
- H04N 19/124: Quantisation
- G06F 9/5027: {the resource being a machine, e.g. CPUs, Servers, Terminals}
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/063: using electronic means
- G06N 3/08: Learning methods
- H04N 19/172: the region being a picture, frame or field
- G06N 3/045: Combinations of networks
- G06N 3/048: Activation functions
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Systems and techniques are described herein for decreasing quantization latency. In some aspects, a process includes determining a first integer data type of data at least one layer of a neural network is configured to process, and determining a second integer data type of data received for processing by the neural network. The second integer data type can be different than the first integer data type. The process further includes determining a ratio between a first size of the first integer data type and a second size of the second integer data type, and scaling parameters of the at least one layer of the neural network using a scaling factor corresponding to the ratio. The process further includes quantize the scaled parameters of the neural network, and inputting the received data to the neural network with the quantized and scaled parameters.

## Claims

### Claim 1

An apparatus for decreasing quantization latency, the apparatus comprising:
a memory;
one or more processors coupled to the memory and configured to:
determine a first integer data type of data at least one layer of a neural network is configured to process;
determine a second integer data type of data received for processing by the neural network, the second integer data type being different than the first integer data type;
determine a ratio between a first size of the first integer data type and a second size of the second integer data type;
scale parameters of the at least one layer of the neural network using a scaling factor corresponding to the ratio;
quantize the scaled parameters of the neural network; and
input the received data to the neural network with the quantized and scaled parameters.

### Claim 2 (depends on 1)

The apparatus of claim 1, further comprising a hardware accelerator configured to implement the neural network using data of the first integer data type.

### Claim 3 (depends on 1)

The apparatus of any one of claim 1, wherein:
the received data includes image data captured by a camera device of the apparatus; and
the neural network is trained to perform one or more image processing operations on the image data.

### Claim 4 (depends on 1)

The apparatus of claim 1, wherein the one or more processors are configured to train the neural network using training data of a floating point data type, wherein training the neural network generates neural network parameters of the floating point data type.

### Claim 5 (depends on 4)

The apparatus of claim 4, wherein the one or more processors are configured to convert the neural network parameters from the floating point data type to the first integer data type.

### Claim 6 (depends on 1)

The apparatus of claim 1, wherein:
the at least one layer of the neural network corresponds to a single layer of the neural network; and
the scaling factor is the ratio between the first size of the first integer data type and the second size of the second integer data type.

### Claim 7 (depends on 1)

The apparatus of claim 1, wherein:
the first size of the first integer data type corresponds to a first number of distinct integers the first integer data type is configured to represent; and
the second size of the second integer data type corresponds to a second number of distinct integers the second integer data type is configured to represent.

### Claim 8 (depends on 1)

The apparatus of claim 1, wherein the at least one layer of the neural network includes a convolutional layer or a deconvolution layer.

### Claim 9 (depends on 1)

The apparatus of claim 1, wherein the at least one layer of the neural network includes a scale layer.

### Claim 10 (depends on 1)

The apparatus of claim 1, wherein the at least one layer of the neural network includes a layer that performs an elementwise operation.

### Claim 11 (depends on 1)

The apparatus of claim 1, wherein the one or more processors are configured to input the received data to the neural network without quantizing the received data.

### Claim 12 (depends on 1)

The apparatus of claim 1, wherein the one or more processors are configured to quantize parameters of one or more additional layers of the neural network.

### Claim 13 (depends on 1)

The apparatus of claim 1, wherein the apparatus includes a mobile device.

### Claim 14 (depends on 1)

The apparatus of claim 1, further comprising a display.

### Claim 15

A method of decreasing quantization latency, the method comprising:
determining a first integer data type of data at least one layer of a neural network is configured to process;
determining a second integer data type of data received for processing by the neural network, the second integer data type being different than the first integer data type;
determining a ratio between a first size of the first integer data type and a second size of the second integer data type;
scaling parameters of the at least one layer of the neural network using a scaling factor corresponding to the ratio;
quantizing the scaled parameters of the neural network; and
inputting the received data to the neural network with the quantized and scaled parameters.

### Claim 16 (depends on 15)

The method of claim 15, further comprising implementing the neural network using a hardware accelerator and data of the first integer data type.

### Claim 17 (depends on 15)

The method of claim 15, wherein:
the received data includes image data captured by a camera device; and
the neural network is trained to perform one or more image processing operations on the image data.

### Claim 18 (depends on 15)

The method of claim 15, further comprising training the neural network using training data of a floating point data type, wherein training the neural network generates neural network parameters of the floating point data type.

### Claim 19 (depends on 18)

The method of claim 18, further comprising converting the neural network parameters from the floating point data type to the first integer data type.

### Claim 20 (depends on 15)

The method of claim 15, wherein:
the at least one layer of the neural network corresponds to a single layer of the neural network; and
the scaling factor is the ratio between the first size of the first integer data type and the second size of the second integer data type.

### Claim 21 (depends on 15)

The method of claim 15, wherein:
the first size of the first integer data type corresponds to a first number of distinct integers the first integer data type is configured to represent; and
the second size of the second integer data type corresponds to a second number of distinct integers the second integer data type is configured to represent.

### Claim 22 (depends on 15)

The method of claim 15, wherein the at least one layer of the neural network includes a convolutional layer or a deconvolution layer.

### Claim 23 (depends on 15)

The method of claim 15, wherein the at least one layer of the neural network includes a scale layer.

### Claim 24 (depends on 15)

The method of claim 15, wherein the at least one layer of the neural network includes a layer that performs an elementwise operation.

### Claim 25 (depends on 15)

The method of claim 15, further comprising inputting the received data to the neural network without quantizing the received data.

### Claim 26 (depends on 15)

The method of claim 15, further comprising quantizing parameters of one or more additional layers of the neural network.

### Claim 27

A non-transitory computer-readable medium having stored thereon instructions that, when executed by one or more processors, cause the one or more processors to:
determine a first integer data type of data at least one layer of a neural network is configured to process;
determine a second integer data type of data received for processing by the neural network, the second integer data type being different than the first integer data type;
determine a ratio between a first size of the first integer data type and a second size of the second integer data type;
scale parameters of the at least one layer of the neural network using a scaling factor corresponding to the ratio;
quantize the scaled parameters of the neural network; and
input the received data to the neural network with the quantized and scaled parameters.

### Claim 28 (depends on 27)

The non-transitory computer-readable medium of claim 27, further comprising instructions that, when executed by one or more processors, cause the one or more processors to implement the neural network using a hardware accelerator and data of the first integer data type.

### Claim 29 (depends on 27)

The non-transitory computer-readable medium of claim 27, wherein:
the received data includes image data captured by a camera device; and
the neural network is trained to perform one or more image processing operations on the image data.

### Claim 30 (depends on 27)

The non-transitory computer-readable medium of claim 27, further comprising instructions that, when executed by one or more processors, cause the one or more processors to train the neural network using training data of a floating point data type, wherein training the neural network generates neural network parameters of the floating point data type.
