# US 2021/0279574 A1 · Method, apparatus, system, storage medium and application for generating quantized neural network

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0279574 A1
- Document ID: US20210279574A1
- Status: Published application
- Kind: A1
- Application number: 17/189,014
- Filed: 2021-03-01
- Published: 2021-09-09
- Earliest claimed: 2020-03-04
- Applicants: Canon Kabushiki Kaisha
- Inventors: Junjie Liu; Tsewei Chen; Dongchao Wen; Wei Tao; Deyu Wang
- Web page: https://patentgrep.com/patent/US20210279574A1

## CPC

- G06N 3/08: Learning methods
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/09: Supervised learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A method of generating a quantized neural network comprises: determining, based on a floating-point weight in a neural network to be quantized, networks which correspond to the floating-point weights and are used for directly outputting quantized weights, respectively; quantizing, using the determined network, the floating-point weight corresponding to the network to obtain a quantized neural network; updating, based on a loss function value obtained via the quantized neural network, the determined network, the floating-point weight and the quantized weight in the quantized neural network.

## Claims

### Claim 1

A method of generating a quantized neural network comprising:
determining, based on floating-point weights in a neural network to be quantized, networks which correspond to the floating-point weights and are used for directly outputting quantized weights, respectively;
quantizing, using the determined network, the floating-point weight corresponding to the network to obtain a quantized neural network; and
updating, based on a loss function value obtained via the quantized neural network, the determined network, the floating-point weight and the quantized weight in the quantized neural network.

### Claim 2 (depends on 1)

The method according to claim 1, wherein, the determined network includes:
a module for convolving floating-point weights; and
a first objective function for constraining an output of the module for convolving the floating-point weights.

### Claim 3 (depends on 2)

The method according to claim 2, wherein the module for convolving the floating-point weights includes:
a first module for converting a dimension of the floating-point weight; and
a second module for converting a dimension of an output of the first module into a dimension of the floating-point weight.

### Claim 4 (depends on 3)

The method according to claim 3, wherein the module for convolving the floating-point weights further includes:
a third module for extracting principal components from the output of the first module,
wherein, the second module is used for converting a dimension of an output of the third module into a dimension of the floating-point weight.

### Claim 5 (depends on 4)

The method according to claim 4, wherein, for one floating-point weight in the neural network to be quantized and the determined network corresponding to the floating-point weight, input shape sizes and numbers of output channels of the first module, the second module and the third module in the network are determined based on a shape size of the floating-point weight.

### Claim 6 (depends on 4)

The method according to claim 4, wherein the first module, the second module and the third module comprise at least one neural network layer, respectively.

### Claim 7 (depends on 2)

The method according to claim 2, wherein, for one floating-point weight in the neural network to be quantized and the determined network corresponding to the floating-point weight, the first objective function in the network preferentially tends elements that can reduce loss of an objective task in the output of the module for convolving the floating-point weights to a quantized weight based on a priority of the elements in the floating-point weight.

### Claim 8 (depends on 1)

The method according to claim 1, wherein, the updating includes:
updating the quantized weight in the quantized neural network based on one loss function value, wherein the loss function value is obtained based on a second objective function for updating the quantized neural network; and
updating the floating-point weight and the determined network based on another loss function value, wherein the loss function value is obtained based on the updated quantized weight and the first objective function.

### Claim 9 (depends on 1)

The method according to claim 1, further comprising:
storing the quantized neural network obtained in the quantization after the update is ended.

### Claim 10 (depends on 9)

The method according to claim 9, wherein, in the storing, the quantized weight in the quantized neural network or the fixed-point weight after the quantized weight is enabled fixed-point are stored.

### Claim 11

An apparatus for generating a quantized neural network, comprising:
a determination unit that determines, based on floating-point weights in a neural network to be quantized, networks which correspond to the floating-point weights and are used for directly outputting quantized weights, respectively;
a quantization unit that quantizes, using the determined network, the floating-point weight corresponding to the network to obtain a quantized neural network; and
an update unit that updates, based on a loss function value obtained via the quantized neural network, the determined network, the floating-point weight and the quantized weight in the quantized neural network.

### Claim 12 (depends on 11)

The apparatus according to claim 11, wherein, the determined network includes:
a module for convolving floating-point weights; and
a first objective function for constraining an output of the module for convolving the floating-point weights.

### Claim 13 (depends on 12)

The apparatus according to claim 12, wherein, for one floating-point weight in the neural network to be quantized and the determined network corresponding to the floating-point weight, the first objective function in the network preferentially tends elements that can reduce loss of an objective task in the output of the module for convolving the floating-point weights to a quantized weight based on a priority of the elements in the floating-point weight.

### Claim 14 (depends on 11)

The apparatus according to claim 11, further comprising:
a storage unit configured to store the quantized neural network obtained by the quantization unit after the operation of the update unit is ended.

### Claim 15

A system for generating a quantized neural network, characterized by comprising:
a first embedded device that determines, based on floating-point weights in a neural network to be quantized, networks which correspond to the floating-point weights and are used for directly outputting quantized weights, respectively;
a second embedded device that quantifies, using a network determined by the first embedded device, the floating-point weight corresponding to the network to obtain a quantized neural network; and
a server that calculates a loss function value via the quantized neural network obtained by the second embedded device, and updates, based on the calculated loss function value, the determined network, the floating-point weight and the quantized weight in the quantized neural network,
wherein the first embedded device, the second embedded device and the server are connected to each other via a network.

### Claim 16

A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, enable to execute generation of a quantized neural network, characterized in that the instructions comprise:
a determination step of determining, based on floating-point weights in a neural network to be quantized, networks which correspond to the floating-point weights and are used for directly outputting quantized weights, respectively;
a quantization step of quantizing, using the determined network, the floating-point weight corresponding to the network to obtain a quantized neural network; and
an update step of updating, based on a loss function value obtained via the quantized neural network, the determined network, the floating-point weight and the quantized weight in the quantized neural network.

### Claim 17

A method of applying a quantized neural network, comprising:
loading a quantized neural network;
inputting, to the quantized neural network, a data set which is required to correspond to a task which can be executed by the quantized neural network;
performing operation on the data set in each layer in the quantized neural network from top to bottom; and
outputting a result.

### Claim 18 (depends on 17)

The method according to claim 17, wherein the loaded quantized neural network is a quantized neural network obtained by a method comprising:
determining, based on floating-point weights in a neural network to be quantized, networks which correspond to the floating-point weights and are used for directly outputting quantized weights, respectively;
quantizing, using the determined network, the floating-point weight corresponding to the network to obtain a quantized neural network; and
updating, based on a loss function value obtained via the quantized neural network, the determined network, the floating-point weight and the quantized weight in the quantized neural network.
