# US 2019/0171927 A1 · Layer-level quantization in neural networks

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0171927 A1
- Document ID: US20190171927A1
- Status: Published application
- Kind: A1
- Application number: 15/833,985
- Filed: 2017-12-06
- Published: 2019-06-06
- Earliest claimed: 2017-12-06
- Assignees: Facebook, Inc.
- Applicants: Facebook, Inc.
- Inventors: Abdulkadir Utku Diril; Jong Soo Park; Nadav Rotem; Mikhail Smelyanskiy
- Web page: https://patentgrep.com/patent/US20190171927A1

## CPC

- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/08: Learning methods
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/048: Activation functions
- G06N 3/0499: Feedforward networks
- G06N 3/063: using electronic means
- G06N 5/046: Forward inferencing; Production systems
- G06N 20/00: Machine learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A method for performing layer-level quantization may include (1) performing an inference of an activation layer of a neural network, (2) storing a first limit value of the activation layer in a data storage system, (3) storing a second limit value of the activation layer in the data storage system, (4) determining a scaling factor based on the first and second limit values, and then (5) applying the scaling factor on a subsequent inference. Various other methods, systems, and devices are also disclosed.

## Claims

### Claim 1

A computing system comprising:
a data storage subsystem; and
a hardware processing unit programmed to:
perform an inference of an activation layer of a neural network;
store a first limit value of the activation layer in the data storage subsystem;
store a second limit value of the activation layer in the data storage subsystem;
determine a scaling factor based on the first and second limit values; and
apply the scaling factor on a subsequent inference.

### Claim 2 (depends on 1)

The computing system of claim 1, wherein the hardware processing unit comprises an accelerator configured to maintain the first and second limit values and the scaling factor in the data storage subsystem.

### Claim 3 (depends on 2)

The computing system of claim 2, wherein the accelerator is further configured to associate the scaling factor with the activation layer.

### Claim 4 (depends on 1)

The computing system of claim 1, further comprising a processing element for determining a minimum value of the activation layer and a maximum value of the activation layer, wherein the first limit value corresponds to the minimum value and the second limit value corresponds to the maximum value.

### Claim 5 (depends on 1)

The computing system of claim 1, wherein applying the scaling factor reduces a bit width needed for at least one arithmetic operation within the neural network.

### Claim 6 (depends on 1)

The computing system of claim 1, wherein the hardware processing unit is further configured to dynamically update the scaling factor.

### Claim 7 (depends on 6)

The computing system of claim 6, wherein the hardware processing unit is further programmed to update the scaling factor until the first limit value and the second limit value stabilize within a predetermined range.

### Claim 8

An accelerator comprising:
a first data storage unit;
a second data storage unit; and
a processing unit configured to:
perform an inference of an activation layer of a neural network;
store a first limit value of the activation layer in the first data storage unit;
store a second limit value of the activation layer in the second data storage unit;
determine a scaling factor based on the first and second limit values; and
apply the scaling factor on a subsequent inference.

### Claim 9 (depends on 8)

The accelerator of claim 8, further comprising a storage subsystem, wherein:
the processing unit is configured to store the scaling factor in the storage subsystem in a manner that associates the scaling factor with the activation layer;
the storage subsystem comprises the first and second data storage units.

### Claim 10 (depends on 8)

The accelerator of claim 8, further comprising a processing element for determining a minimum value of the activation layer and a maximum value of the activation layer, wherein the first limit value corresponds to the minimum value and the second limit value corresponds to the maximum value.

### Claim 11 (depends on 8)

The accelerator of claim 8, wherein applying the scaling factor reduces a bit width needed for at least one arithmetic operation within the neural network.

### Claim 12 (depends on 8)

The accelerator of claim 8, wherein the processing unit is configured to dynamically update the scaling factor.

### Claim 13 (depends on 12)

The accelerator of claim 12, wherein the processing unit is configured to update the scaling factor until the first limit value and the second limit value stabilize within a predetermined range.

### Claim 14

A method comprising:
performing an inference of an activation layer of a neural network;
storing a first limit value of the activation layer in a data storage system;
storing a second limit value of the activation layer in the data storage system;
determining a scaling factor based on the first and second limit values; and
applying the scaling factor on a subsequent inference.

### Claim 15 (depends on 14)

The method of claim 14, further comprising performing, before or after applying the scaling factor, an offset operation.

### Claim 16 (depends on 15)

The method of claim 15, further comprising associating the scaling factor with the activation layer.

### Claim 17 (depends on 14)

The method of claim 14, further comprising determining a minimum value of the activation layer and a maximum value of the activation layer, wherein the first limit value corresponds to the minimum value and the second limit value corresponds to the maximum value.

### Claim 18 (depends on 14)

The method of claim 14, wherein applying the scaling factor reduces a bit width needed for at least one arithmetic operation within the neural network.

### Claim 19 (depends on 14)

The method of claim 14, further comprising periodically updating the scaling factor.

### Claim 20 (depends on 19)

The method of claim 19, further comprising updating the scaling factor until the first limit value and the second limit value stabilize within a predetermined range.
