Back to results
US 12,591,771 B2Granted
Dynamic quantization for energy efficient deep learning
AssigneeQUALCOMM Incorporated
InventorsRandy Ardywibowo, Venkata Ravi Kiran Dayana, Hau Hwang
Life of this patent
Estimating the term…
Abstract
A method performed by a deep neural network (DNN) includes receiving, at a layer of the DNN during an inference stage, a layer input comprising content associated with a DNN input received at the DNN. The method also includes quantizing one or more parameters of a plurality of parameters associated with the layer based on the content of the layer input. The method further includes performing a task corresponding to the DNN input, the task performed with the one or more one quantized parameters.
Claims28
A method performed by a deep neural network (DNN), comprising:
- receiving, at a layer of the DNN during an inference stage, a layer input comprising content associated with a DNN input received at the DNN, each output channel of the layer having an original bit-width prior to receiving the layer input;
- quantizing one or more parameters of a respective plurality of parameters associated with each output channel of one or more output channels of the layer based on the content of the layer input;
- reducing the respective original bit-width for each output channel of the one or more output channels of the layer to generate a respective adjusted bit-width for each output channel of the one or more output channels in accordance with quantizing the one or more parameters, the original bit-width being associated with the one or more parameters, a quantity of bits processed via one or more processors executing the DNN corresponding to a size of the adjusted bit-width, such that one or more of a compute power of the one or more processors or a memory bandwidth of the one or more processors is reduced in accordance with reducing the respective original bit-width for each output channel of the one or more output channels;
- and
- performing a task corresponding to the DNN input, the task performed via the one or more processors in accordance with the adjusted bit-width.
2 more elements
An apparatus for implementing a deep neural network (DNN), comprising:
- one or more processors;
- one or more memories coupled with the one or more processors;
- and
- instructions stored in the one or more memories and operable, when executed by the one or more processors, to cause the apparatus to:
- receive, at a layer of the DNN during an inference stage, a layer input comprising content associated with a DNN input received at the DNN, each output channel of the layer having an original bit-width prior to receiving the layer input;
- quantize one or more parameters of a respective plurality of parameters associated with each output channel of one or more output channels of the layer based on the content of the layer input;
- reduce the respective original bit-width for each output channel of the one or more output channels of the layer to generate a respective adjusted bit-width for each output channel of the one or more output channels in accordance with quantizing the one or more parameters, the original bit-width being associated with the one or more parameters, a quantity of bits processed via the one or more processors executing the DNN corresponding to a size of the adjusted bit-width, such that one or more of a compute power of the one or more processors or a memory bandwidth of the one or more processors is reduced in accordance with reducing the respective original bit-width for each output channel of the one or more output channels;
- and
- perform a task corresponding to the DNN input, the task performed via the one or more processors in accordance with the adjusted bit-width.
6 more elements
A non-transitory computer-readable medium having program code recorded thereon for a deep neural network (DNN), the program code executed by one or more processors and comprising:
- program code to receive, at a layer of the DNN during an inference stage, a layer input comprising content associated with a DNN input received at the DNN, each output channel of the layer having an original bit-width prior to receiving the layer input;
- program code to quantize one or more parameters of a respective plurality of parameters associated with each output channel of one or more output channels of the layer based on the content of the layer input;
- program code to reduce the respective original bit-width for each output channel of the one or more output channels of the layer to generate a respective adjusted bit-width for each output channel of the one or more output channels in accordance with quantizing the one or more parameters, the original bit-width being associated with the one or more parameters, a quantity of bits processed via the one or more processors executing the DNN corresponding to a size of the adjusted bit-width, such that one or more of a compute power of the one or more processors or a memory bandwidth of the one or more processors is reduced in accordance with reducing the respective original bit-width for each output channel of the one or more output channels;
- and
- program code to perform a task corresponding to the DNN input, the task performed via the one or more processors in accordance with the adjusted bit-width.
2 more elements
An apparatus implementing a deep neural network (DNN), comprising:
- means for receiving, at a layer of the DNN during an inference stage, a layer input comprising content associated with a DNN input received at the DNN, each output channel of the layer having an original bit-width prior to receiving the layer input;
- means for quantizing one or more parameters of a respective plurality of parameters associated with each output channel of one or more output channels of the layer based on the content of the layer input;
- means for reducing the respective original bit-width for each output channel of the one or more output channels of the layer to generate a respective adjusted bit-width for each output channel of the one or more output channels in accordance with quantizing the one or more parameters, the original bit-width being associated with the one or more parameters, a quantity of bits processed via one or more processors executing the DNN corresponding to a size of the adjusted bit-width, such that one or more of a compute power of the one or more processors or a memory bandwidth of the one or more processors is reduced in accordance with reducing the respective original bit-width for each output channel of the one or more output channels;
- and
- means for performing a task corresponding to the DNN input, the task performed via the one or more processors in accordance with the adjusted bit-width.
2 more elements
Dependencies are inferred from the claim text.