Dynamic quantization of neural networks
AssigneeIntel Corporation
InventorMichael E. Deisher
Life of this patent
Estimating the term…
Abstract
An apparatus for applying dynamic quantization of a neural network is described herein. The apparatus includes a scaling unit and a quantizing unit. The scaling unit is to calculate an initial desired scale factors of a plurality of inputs, weights and a bias and apply the input scale factor to a summation node. Also, the scaling unit is to determine a scale factor for a multiplication node based on the desired scale factors of the inputs and select a scale factor for an activation function and an output node. The quantizing unit is to dynamically requantize the neural network by traversing a graph of the neural network.
Claims26
(canceled)
A system comprising:
- a memory device configured to store instructions;
- one or more processors including a hardware accelerator, the one or more processors configured to execute the instructions, wherein the instructions cause the one or more processors to:
- perform matrix multiplication operations via the hardware accelerator to generate a plurality of first values;
- determine a scale factor based on a maximum value within a window of the plurality of first values;
- train a neural network based on a plurality of second values scaled by the scale factor;
- and
- quantize respective ones of the scaled second values.
At least one non-transitory machine-readable medium comprising machine-readable instructions to cause at least one programmable circuit to at least:
- cause a hardware accelerator to perform matrix multiplication operations to generate a plurality of first values;
- determine a scale factor based on a maximum value within a window of the plurality of first values;
- train a neural network based on a plurality of second values scaled by the scale factor;
- and
- quantize respective ones of the scaled second values.
A system comprising:
- a memory device;
- instructions;
- one or more processor circuits including a hardware accelerator, the one or more processor circuits to be programmed by the instructions to:
- perform matrix multiplication operations via the hardware accelerator to generate a plurality of first values;
- determine a scale factor based on a maximum value within a window of the plurality of first values;
- train a neural network based on a plurality of second values scaled by the scale factor;
- and
- quantize respective ones of the scaled second values.
Dependencies are inferred from the claim text.