Back to results
US 11,961,000 B2Granted
Lossy layer compression for dynamic scaling of deep neural network processing
AssigneeQUALCOMM Incorporated
InventorWesley James Holland
Life of this patent
Estimating the term…
Abstract
An apparatus of operating a neural network is configured to compress one or more of activations or weights in one or more layer of the neural network. The activations and/or weights may be compressed based on a compression ratio or a system event. The system event may be a bandwidth condition, a power condition, a debug condition, a thermal condition or the like. The apparatus may operate the neural network to compute an inference based on the compressed activations or the compressed weights.
Claims20
A computer-implemented method of operating a neural network, comprising:
- generating, at a processor, a compression mapping table based at least in part on at least one of a sparsity estimate or a loss threshold for a selected bandwidth, power, or thermal profile, wherein the compression map table comprises at least one column indicating a specific compression ratio for both a weight and an activation associated with each different system event and a plurality of table rows, each row of the plurality of tables rows corresponding to a specific layer of the neural network;
- determining, at the processor, a first compression ratio for a first layer of the neural network based on a first threshold associated with a system event using the compression mapping table and detection of a layer type of the first layer in the neural network;
- compressing, via the processor, at least one of activations or weights in the first layer of the neural network based at least in part on the first compression ratio and detection of the system event to produce at least one of compressed activations or compressed weights;
- generating compression metadata indicating how data was compressed;
- and
- operating, via the processor, the neural network to compute an inference based on the at least one of the compressed activations or the compressed weights.
3 more elements
An apparatus of operating a neural network, comprising:
- a memory;
- and
- at least one processor coupled to the memory, the at least one processor being configured to:
- generate a compression mapping table based at least in part on at least one of a sparsity estimate or a loss threshold for a selected bandwidth, power, or thermal profile, wherein the compression mapping table comprising at least one column indicating a specific compression ratio for both a weight and an activation associated with each different system event and a plurality of table rows, each row of the plurality of tables rows corresponding to a specific layer of the neural network;
- determine a compression ratio for a first layer of the neural network based on a threshold associated with a system event using the compression mapping table and detection of a layer type of the first layer in the neural network;
- compress at least one of activations or weights in the first layer of the neural network based at least in part on the compression ratio and detection of the system event to produce at least one of compressed activations or compressed weights;
- generate compression metadata indicating how data was compressed;
- and
- operate the neural network to compute an inference based on the at least one of the compressed activations or the compressed weights.
6 more elements
An apparatus for operating a neural network, comprising:
- means for generating, at a processor, a compression mapping table based at least in part on at least one of a sparsity estimate or a loss threshold for a selected bandwidth, power, or thermal profile, wherein the compression map table comprises at least one column indicating a specific compression ratio for both a weight and an activation associated with each different system event and a plurality of table rows, each row of the plurality of tables rows corresponding to a specific layer of the neural network;
- means for determining a compression ratio for a first layer of the neural network based on a threshold associated with a system event using the compression mapping table and detection of a layer type of the first layer in the neural network;
- means for compressing at least one of activations or weights in the first layer of the neural network based at least in part on the compression ratio and detection of the system event to produce at least one of compressed activations or compressed weights;
- means for generating compression metadata indicating how data was compressed;
- and
- means for operating the neural network to compute an inference based on the at least one of the compressed activations or the compressed weights.
3 more elements
A non-transitory computer-readable medium storing computer executable code for operating a neural network, comprising code to:
- generate a compression mapping table based at least in part on at least one of a sparsity estimate or a loss threshold for a selected bandwidth, power, or thermal profile, wherein the compression mapping table comprising at least one column indicating a specific compression ratio for both a weight and an activation associated with each different system event and a plurality of table rows, each row of the plurality of tables rows corresponding to a specific layer of the neural network;
- determine a compression ratio for a first layer of the neural network based on a threshold associated with a system event using the compression mapping table and detection of a layer type of the first layer in the neural network;
- compress at least one of activations or weights in the first layer of the neural network based at least in part on the compression ratio and detection of the system event to produce at least one of compressed activations or compressed weights;
- generate compression metadata indicating how data was compressed;
- and
- operate the neural network to compute an inference based on the at least one of the compressed activations or the compressed weights.
3 more elements
Dependencies are inferred from the claim text.