# US 2026/0017519 A1 · Neural network compression

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0017519 A1
- Document ID: US20260017519A1
- Status: Published application
- Kind: A1
- Application number: 19/335,336
- Filed: 2025-09-22
- Published: 2026-01-15
- Earliest claimed: 2018-12-18
- Assignees: Movidius Limited
- Applicants: Movidius Ltd.
- Inventors: Xiaofan Xu; Mi Sun Park; Cormac M. Brick
- Web page: https://patentgrep.com/patent/US20260017519A1

## CPC

- G06N 3/063: using electronic means
- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/047: Probabilistic or stochastic networks
- G06N 3/048: Activation functions
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- H03M 7/702: {Software}
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- H03M: CODING; DECODING; CODE CONVERSION IN GENERAL (using fluidic means F15C4/00; optical analogue/digital converters G02F7/00; coding, decoding or code conversion, specially adapted for particular applications, see the relevant subclasses, e.g. G01D, G01R, G06F, G06T, G09G, G10L, G11B, G11C, H04B, H04L, H04M, H04N; ciphering or deciphering for cryptography or other purposes involving the need for secrecy G09C)

## Abstract

A neural network model is trained, where the training includes multiple training iterations. Weights of a particular layer of the neural network are pruned during a forward pass of a particular one of the training iterations. During the same forward pass of the particular training iteration, values of weights of the particular layer are quantized to determine a quantized-sparsified subset of weights for the particular layer. A compressed version of the neural network model is generated from the training based at least in part on the quantized-sparsified subset of weights.

## Claims

### Claim 1

One or more non-transitory computer-readable media storing instructions executable to perform operations for training a neural network model, the operations comprising:
pruning a first group of weights in a set of weights of a layer in the neural network model;
quantizing a second group of weights in the set of weights of the layer to produce a new set of weights of the layer, the new set of weights comprising one or more quantized weights and one or more full-precision weights for the layer, the one or more quantized weights having a lower data precision than the one or more full-precision weights;
updating the new set of weights by modifying one or more values in the new set of weights based on a training dataset and a loss function; and
generate a compressed version of the neural network model, wherein the compressed version of the neural network model comprising the layer with the updated new set of weights.

### Claim 2 (depends on 1)

The one or more non-transitory computer-readable media of claim 1, wherein modifying one or more values in the new set of weights comprises:
modifying a value of a full-precision weight in the new set of weights.

### Claim 3 (depends on 1)

The one or more non-transitory computer-readable media of claim 1, wherein updating the new set of weights comprises:
quantizing a gradient value during a backward pass, the gradient value determined based on the loss function; and
modifying the one or more values in the new set of weights based on the quantized gradient value.

### Claim 4 (depends on 1)

The one or more non-transitory computer-readable media of claim 1, wherein updating the new set of weights comprises:
keeping one or more values of the one or more sparse-quantized weights the same.

### Claim 5 (depends on 1)

The one or more non-transitory computer-readable media of claim 1, wherein the second group of weights are quantized after the first group of weights are pruned.

### Claim 6 (depends on 1)

The one or more non-transitory computer-readable media of claim 1, wherein the operations further comprise:
before updating the new set of weights, quantizing one or more activations of the layer.

### Claim 7 (depends on 1)

The one or more non-transitory computer-readable media of claim 1, wherein the operations further comprise:
selecting the first group of weights by determining that each weight in the first group of weights has a value that is lower than a threshold.

### Claim 8 (depends on 7)

The one or more non-transitory computer-readable media of claim 7, wherein the operations further comprise:
determining the threshold for the layer;
determining a different threshold for a different layer in the neural network model; and
pruning one or more weights of the different layer based on the different threshold.

### Claim 9 (depends on 7)

The one or more non-transitory computer-readable media of claim 7, wherein the operations further comprise:
determining the threshold for a training iteration in a training process, wherein the compressed version of the neural network model is generated within the training iteration;
determining a different threshold for another training iteration in the training process; and
generating another compressed version of the neural network model based on the different threshold within the another training iteration.

### Claim 10

An apparatus, comprising:
a computer processor for executing computer program instructions; and
a non-transitory computer-readable memory storing computer program instructions executable by the computer processor to perform operations for training a neural network model, the operations comprising:
pruning a first group of weights in a set of weights of a layer in the neural network model,
quantizing a second group of weights in the set of weights of the layer to produce a new set of weights of the layer, the new set of weights comprising one or more quantized weights and one or more full-precision weights for the layer, the one or more quantized weights having a lower data precision than the one or more full-precision weights,
updating the new set of weights by modifying one or more values in the new set of weights based on a training dataset and a loss function, and
generate a compressed version of the neural network model, wherein the compressed version of the neural network model comprising the layer with the updated new set of weights.

### Claim 11 (depends on 10)

The apparatus of claim 10, wherein modifying one or more values in the new set of weights comprises:
modifying a value of a full-precision weight in the new set of weights.

### Claim 12 (depends on 10)

The apparatus of claim 10, wherein updating the new set of weights comprises:
quantizing a gradient value during a backward pass, the gradient value determined based on the loss function; and
modifying the one or more values in the new set of weights based on the quantized gradient value.

### Claim 13 (depends on 10)

The apparatus of claim 10, wherein updating the new set of weights comprises:
keeping one or more values of the one or more sparse-quantized weights the same.

### Claim 14 (depends on 10)

The apparatus of claim 10, wherein the second group of weights are quantized after the first group of weights are pruned.

### Claim 15 (depends on 10)

The apparatus of claim 10, wherein the operations further comprise:
before updating the new set of weights, quantizing one or more activations of the layer.

### Claim 16 (depends on 10)

The apparatus of claim 10, wherein the operations further comprise:
selecting the first group of weights by determining that each weight in the first group of weights has a value that is lower than a threshold.

### Claim 17 (depends on 16)

The apparatus of claim 16, wherein the operations further comprise:
determining the threshold for a training iteration in a training process, wherein the compressed version of the neural network model is generated within the training iteration;
determining a different threshold for another training iteration in the training process; and
generating another compressed version of the neural network model based on the different threshold within the another training iteration.

### Claim 18

A method for training a neural network model, the method comprising:
pruning a first group of weights in a set of weights of a layer in the neural network model, and
quantizing a second group of weights in the set of weights of the layer to produce a new set of weights of the layer, the new set of weights comprising one or more quantized weights and one or more full-precision weights for the layer, the one or more quantized weights having a lower data precision than the one or more full-precision weights;
updating the new set of weights by modifying one or more values in the new set of weights based on a training dataset and a loss function; and
generate a compressed version of the neural network model, wherein the compressed version of the neural network model comprising the layer with the updated new set of weights.

### Claim 19 (depends on 18)

The method of claim 18, wherein modifying one or more values in the new set of weights comprises:
modifying a value of a full-precision weight in the new set of weights.

### Claim 20 (depends on 18)

The method of claim 18, wherein updating the new set of weights comprises:
quantizing a gradient value during a backward pass, the gradient value determined based on the loss function; and
modifying the one or more values in the new set of weights based on the quantized gradient value.
