Neural network compression
AssigneeMovidius Limited
InventorsXiaofan Xu, Mi Sun Park, Cormac M. Brick
Life of this patent
Estimating the term…
Abstract
A neural network model is trained, where the training includes multiple training iterations. Weights of a particular layer of the neural network are pruned during a forward pass of a particular one of the training iterations. During the same forward pass of the particular training iteration, values of weights of the particular layer are quantized to determine a quantized-sparsified subset of weights for the particular layer. A compressed version of the neural network model is generated from the training based at least in part on the quantized-sparsified subset of weights.
Claims20
One or more non-transitory computer-readable media storing instructions executable to perform operations for training a neural network model, the operations comprising:
- pruning a first group of weights in a set of weights of a layer in the neural network model;
- quantizing a second group of weights in the set of weights of the layer to produce a new set of weights of the layer, the new set of weights comprising one or more quantized weights and one or more full-precision weights for the layer, the one or more quantized weights having a lower data precision than the one or more full-precision weights;
- updating the new set of weights by modifying one or more values in the new set of weights based on a training dataset and a loss function;
- and
- generate a compressed version of the neural network model, wherein the compressed version of the neural network model comprising the layer with the updated new set of weights.
An apparatus, comprising:
- a computer processor for executing computer program instructions;
- and
- a non-transitory computer-readable memory storing computer program instructions executable by the computer processor to perform operations for training a neural network model, the operations comprising:
- pruning a first group of weights in a set of weights of a layer in the neural network model,
- quantizing a second group of weights in the set of weights of the layer to produce a new set of weights of the layer, the new set of weights comprising one or more quantized weights and one or more full-precision weights for the layer, the one or more quantized weights having a lower data precision than the one or more full-precision weights,
- updating the new set of weights by modifying one or more values in the new set of weights based on a training dataset and a loss function, and
- generate a compressed version of the neural network model, wherein the compressed version of the neural network model comprising the layer with the updated new set of weights.
A method for training a neural network model, the method comprising:
- pruning a first group of weights in a set of weights of a layer in the neural network model, and
- quantizing a second group of weights in the set of weights of the layer to produce a new set of weights of the layer, the new set of weights comprising one or more quantized weights and one or more full-precision weights for the layer, the one or more quantized weights having a lower data precision than the one or more full-precision weights;
- updating the new set of weights by modifying one or more values in the new set of weights based on a training dataset and a loss function;
- and
- generate a compressed version of the neural network model, wherein the compressed version of the neural network model comprising the layer with the updated new set of weights.
Dependencies are inferred from the claim text.