Layer-level quantization in neural networks
AssigneeFacebook, Inc.
InventorsAbdulkadir Utku Diril, Jong Soo Park, Nadav Rotem, Mikhail Smelyanskiy
Life of this patent
Estimating the term…
Abstract
A method for performing layer-level quantization may include (1) performing an inference of an activation layer of a neural network, (2) storing a first limit value of the activation layer in a data storage system, (3) storing a second limit value of the activation layer in the data storage system, (4) determining a scaling factor based on the first and second limit values, and then (5) applying the scaling factor on a subsequent inference. Various other methods, systems, and devices are also disclosed.
Claims20
A computing system comprising:
- a data storage subsystem;
- and
- a hardware processing unit programmed to:
- perform an inference of an activation layer of a neural network;
- store a first limit value of the activation layer in the data storage subsystem;
- store a second limit value of the activation layer in the data storage subsystem;
- determine a scaling factor based on the first and second limit values;
- and
- apply the scaling factor on a subsequent inference.
An accelerator comprising:
- a first data storage unit;
- a second data storage unit;
- and
- a processing unit configured to:
- perform an inference of an activation layer of a neural network;
- store a first limit value of the activation layer in the first data storage unit;
- store a second limit value of the activation layer in the second data storage unit;
- determine a scaling factor based on the first and second limit values;
- and
- apply the scaling factor on a subsequent inference.
A method comprising:
- performing an inference of an activation layer of a neural network;
- storing a first limit value of the activation layer in a data storage system;
- storing a second limit value of the activation layer in the data storage system;
- determining a scaling factor based on the first and second limit values;
- and
- applying the scaling factor on a subsequent inference.
Dependencies are inferred from the claim text.