# US 12,762,397 B2 · Framework for compression-aware training of neural networks

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,762,397 B2
- Document ID: US12762397B2
- Status: Granted
- Kind: B2
- Application number: 17/949,082
- Filed: 2022-09-20
- Published: 2024-03-21
- Granted: 2026-10-13
- Earliest claimed: 2022-09-20
- Assignees: Advanced Micro Devices, Inc.; ATI Technologies ULC
- Applicants: Advanced Micro Devices, Inc.; ATI Technologies ULC
- Inventors: Mehdi Saeedi; Ian Charles Colbert; Ihab M. A. Amer
- Web page: https://patentgrep.com/patent/US12762397B2

## CPC

- G06N 3/08: Learning methods
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Methods and devices are provided for processing data using a neural network. Activations from a previous layer of the neural network are received by a layer of the neural network. Weighted values, to be applied to values of elements of the activations, are determined based on a spatial correlation of the elements and a task error output by the layer. The weighted values are applied to the values of the elements and a combined error is determined based on the task error and the spatial correlation.

## Claims

### Claim 1

A method for reducing off-chip memory bandwidth and power consumed when processing data using a neural network, the method comprising:
during an inference pass:
receiving, by a processor, activations from a previous layer of the neural network;
retrieving, by the processor, first weighted values determined during training of the neural network based on a combined error, wherein the combined error was determined during the training based on a task error and a spatial correlation of elements of activations generated during the training;
applying, by the processor, the first weighted values to values of elements of the activations received from the previous layer to generate intermediate activations; and
before writing the intermediate activations to off-chip memory, reducing, by the processor, off-chip memory bandwidth consumed by compressing the intermediate activations using a delta-compression engine.

### Claim 2 (depends on 1)

The method of claim 1, wherein the combined error is further
determined based on a second spatial correlation of elements of activations from a next layer of the neural network.

### Claim 3 (depends on 2)

The method of claim 2,
wherein the combined error is determined based on a spatial-correlation penalty determined from a combination of the spatial correlation and the second spatial correlation.

### Claim 4 (depends on 3)

The method of claim 3,
wherein the combined error is determined based on a weighted combination of the task error and the spatial-correlation penalty.

### Claim 5 (depends on 1)

The method of claim 1, wherein a portion of the activations corresponds to a layer of the neural network.

### Claim 6 (depends on 1)

The method of claim 1, wherein the first weighted values were updated during the training based on the combined error.

### Claim 7 (depends on 5)

The method of claim 5, wherein the spatial correlation is determined from a variance of values of the elements of the activations generated during the training.

### Claim 8 (depends on 5)

The method of claim 5, wherein the spatial correlation is determined from a mean value of the elements of the activations generated during the training.

### Claim 9

A device for reducing off-chip memory bandwidth and power consumed when processing data using a neural network, the device comprising:
on-chip memory;
an external memory interface coupled to off-chip memory;
a delta-compression engine configured to compress intermediate activations before the intermediate activations are written to the off-chip memory; and
one or more processors configured to:
during an inference pass:
receive activations from a previous layer of the neural network;
retrieve first weighted values determined during training of the neural network based on a combined error, wherein the combined error was determined during the training based on a task error and a spatial correlation of elements of activations generated during the training;
apply the first weighted values to values of elements of the activations received from the previous layer to generate the intermediate activations;
reduce the off-chip memory bandwidth consumed by invoking the delta-compression engine to compress the intermediate activations; and
write the compressed intermediate activations across the external memory interface to the off-chip memory.

### Claim 10 (depends on 9)

The device of claim 9, wherein the
combined error is further determined based on a second spatial correlation of elements of activations from a next layer of the neural network.

### Claim 11 (depends on 10)

The device of claim 10, wherein the
combined error is determined based on a spatial-correlation penalty determined from a combination of the spatial correlation and the second spatial correlation penalty.

### Claim 12 (depends on 11)

The device of claim 11, wherein the
combined error is determined based on a weighted combination of the task error and the spatial-correlation penalty.

### Claim 13 (depends on 9)

The device of claim 9, wherein the first weighted values were updated during the training based on the combined error.

### Claim 14 (depends on 9)

The device of claim 9, wherein the spatial correlation is determined from a variance of values of the elements of the activations generated during the training.

### Claim 15 (depends on 9)

The device of claim 9, wherein the spatial correlation is determined from a mean value of the elements of the activations generated during the training.

### Claim 16 (depends on 9)

The device of claim 9, further comprising a display device, wherein data processed using the neural network is displayed as images on the display device.

### Claim 17

A non-transitory computer-readable medium storing instructions for reducing off-chip memory bandwidth and power consumed when processing data using a neural network, the instructions, when executed by one or more processors, cause the one or more processors to perform operations comprising:
during an inference pass:
receiving activations from a previous layer of the neural network;
retrieving first weighted values determined during training of the neural network based on a combined error, wherein the combined error was determined during the training based on a task error and a spatial correlation of elements of activations generated during the training;
applying the first weighted values to values of elements of the activations received from the previous layer to generate intermediate activations; and
before writing the intermediate activations to off-chip memory, reducing off-chip memory bandwidth consumed by compressing the intermediate activations using a delta-compression engine.

### Claim 18 (depends on 1)

The method of claim 1, wherein the spatial correlation is calculated by measuring similarity between values of elements of a portion of the activations generated during the training.

### Claim 19 (depends on 9)

The device of claim 9, wherein the spatial correlation is calculated by measuring similarity between values of elements of a portion of the activations generated during the training.