# US 2021/0241110 A1 · Online adaptation of neural network compression using weight masking

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0241110 A1
- Document ID: US20210241110A1
- Status: Published application
- Kind: A1
- Application number: 16/777,200
- Filed: 2020-01-30
- Published: 2021-08-05
- Earliest claimed: 2020-01-30
- Assignees: EMC IP Holding Company LLC
- Applicants: EMC IP Holding Company LLC
- Inventors: Tiago Salviano Calmon; Vinicius Michel Gottin; Paulo Abelha Ferreira
- Web page: https://patentgrep.com/patent/US20210241110A1

## CPC

- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Dynamic adapting neural networks. A latency of a neural network, such as time to inference, is controlled by dynamically compressing/decompressing the neural network. The level of compression or the compression ratio is based on a relationship between the latency and the desired service level. The compression ratio and thus the level of compression can be adjusted until the latency complies with a required latency. A minimum level of accuracy is maintained such that catastrophic forgetting does not occur in the neural network.

## Claims

### Claim 1

A method for dynamically adapting a neural network, the method comprising:
determining a latency associated with the neural network;
determining a current compression level applied to the neural network;
determining a relation parameter that relates the latency with the current compression level;
adjusting a compression ratio based on the relation parameter; and
applying the compression ratio to the neural network.

### Claim 2 (depends on 1)

The method of claim 1, further comprising adjusting the compression ratio such that the latency is less than or equal to a required latency.

### Claim 3 (depends on 2)

The method of claim 2, further comprising adjusting the compression ratio without causing catastrophic forgetting in the neural network.

### Claim 4 (depends on 1)

The method of claim 1, wherein applying the compression ratio includes compressing the neural network or re-enlarging the neural network.

### Claim 5 (depends on 1)

The method of claim 1, further comprising applying the compression ratio by masking weights in the neural network.

### Claim 6 (depends on 1)

The method of claim 1, further comprising applying the compression ratio by masking nodes in the neural network.

### Claim 7 (depends on 1)

The method of claim 1, further comprising applying a masking parameter that determines whether an inference procedure is or is not performed in a given weight node of the neural network.

### Claim 8 (depends on 1)

The method of claim 1, further comprising configuring the neural network to accept a masking parameter.

### Claim 9 (depends on 1)

The method of claim 1, further comprising repeatedly adjusting the compression ratio based on the latency and the current compression level, wherein the latency and the current compression level are determined periodically or continuously.

### Claim 10 (depends on 1)

The method of claim 1, further comprising determining the relation parameter using recursive least squares or a learning algorithm.

### Claim 11

A non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to perform operations comprising:
determining a latency associated with the neural network;
determining a current compression level applied to the neural network;
determining a relation parameter that relates the latency with the current compression level;
adjusting a compression ratio based on the relation parameter; and
applying the compression ratio to the neural network.

### Claim 12 (depends on 11)

The non-transitory storage medium of claim 11, the operations further comprising adjusting the compression ratio such that the latency is less than or equal to a required latency.

### Claim 13 (depends on 12)

The non-transitory storage medium of claim 12, the operations further comprising adjusting the compression ratio without causing catastrophic forgetting in the neural network.

### Claim 14 (depends on 11)

The non-transitory storage medium of claim 11, wherein applying the compression ratio includes compressing the neural network or re-enlarging the neural network.

### Claim 15 (depends on 11)

The non-transitory storage medium of claim 11, the operations further comprising applying the compression ratio by masking weights in the neural network.

### Claim 16 (depends on 11)

The non-transitory storage medium of claim 11, the operations further comprising applying the compression ratio by masking nodes in the neural network.

### Claim 17 (depends on 11)

The non-transitory storage medium of claim 11, the operations further comprising applying a masking parameter that determines whether an inference procedure is or is not performed in a given weight node of the neural network.

### Claim 18 (depends on 11)

The non-transitory storage medium of claim 11, the operations further comprising configuring the neural network to accept a masking parameter.

### Claim 19 (depends on 11)

The non-transitory storage medium of claim 11, the operations further comprising repeatedly adjusting the compression ratio based on the latency and the current compression level, wherein the latency and the current compression level are determined periodically or continuously.

### Claim 20 (depends on 11)

The non-transitory storage medium of claim 11, the operations further comprising determining the relation parameter using recursive least squares or a learning algorithm.
