Framework for compression-aware training of neural networks
AssigneesAdvanced Micro Devices, Inc., ATI Technologies ULC
InventorsMehdi Saeedi, Ian Charles Colbert, Ihab M. A. Amer
Life of this patent
Estimating the term…
Abstract
Methods and devices are provided for processing data using a neural network. Activations from a previous layer of the neural network are received by a layer of the neural network. Weighted values, to be applied to values of elements of the activations, are determined based on a spatial correlation of the elements and a task error output by the layer. The weighted values are applied to the values of the elements and a combined error is determined based on the task error and the spatial correlation.
Claims19
A method for reducing off-chip memory bandwidth and power consumed when processing data using a neural network, the method comprising:
- during an inference pass:
- receiving, by a processor, activations from a previous layer of the neural network;
- retrieving, by the processor, first weighted values determined during training of the neural network based on a combined error, wherein the combined error was determined during the training based on a task error and a spatial correlation of elements of activations generated during the training;
- applying, by the processor, the first weighted values to values of elements of the activations received from the previous layer to generate intermediate activations;
- and
- before writing the intermediate activations to off-chip memory, reducing, by the processor, off-chip memory bandwidth consumed by compressing the intermediate activations using a delta-compression engine.
A device for reducing off-chip memory bandwidth and power consumed when processing data using a neural network, the device comprising:
- on-chip memory;
- an external memory interface coupled to off-chip memory;
- a delta-compression engine configured to compress intermediate activations before the intermediate activations are written to the off-chip memory;
- and
- one or more processors configured to:
- during an inference pass:
- receive activations from a previous layer of the neural network;
- retrieve first weighted values determined during training of the neural network based on a combined error, wherein the combined error was determined during the training based on a task error and a spatial correlation of elements of activations generated during the training;
- apply the first weighted values to values of elements of the activations received from the previous layer to generate the intermediate activations;
- reduce the off-chip memory bandwidth consumed by invoking the delta-compression engine to compress the intermediate activations;
- and
- write the compressed intermediate activations across the external memory interface to the off-chip memory.
A non-transitory computer-readable medium storing instructions for reducing off-chip memory bandwidth and power consumed when processing data using a neural network, the instructions, when executed by one or more processors, cause the one or more processors to perform operations comprising:
- during an inference pass:
- receiving activations from a previous layer of the neural network;
- retrieving first weighted values determined during training of the neural network based on a combined error, wherein the combined error was determined during the training based on a task error and a spatial correlation of elements of activations generated during the training;
- applying the first weighted values to values of elements of the activations received from the previous layer to generate intermediate activations;
- and
- before writing the intermediate activations to off-chip memory, reducing off-chip memory bandwidth consumed by compressing the intermediate activations using a delta-compression engine.
Dependencies are inferred from the claim text.