Model compression for neural networks
AssigneeInternational Business Machines Corporation
InventorsSURIYA T SKARIAH, PRASHANSHA GUPTA, RAHUL M RAO
Life of this patent
Estimating the term…
Abstract
Model compression for neural networks, including: determining, during training of a neural network model, for each of a plurality of training iterations, a corresponding value for a plurality of weights; calculating, for each weight of the plurality of weights and based on the corresponding value for each of the plurality of weights, an importance metric; calculating a plurality of quantized weights by binning the plurality of weights based on the importance metric; and updating the neural network model based on the plurality of quantized weights.
Claims20
A method comprising:
- determining, during training of a neural network model, for each of a plurality of training iterations, a corresponding value for a plurality of weights;
- calculating, for each weight of the plurality of weights and based on the corresponding value for each of the plurality of weights, an importance metric;
- calculating a plurality of quantized weights by binning the plurality of weights based on the importance metric;
- and
- updating the neural network model based on the plurality of quantized weights.
An apparatus comprising:
- a memory;
- and
- a processing device operatively coupled to the memory, the processing device configured to:
- determine, during training of a neural network model, for each of a plurality of training iterations, a corresponding value for a plurality of weights;
- calculate, for each weight of the plurality of weights and based on the corresponding value for each of the plurality of weights, an importance metric;
- calculate a plurality of quantized weights by binning the plurality of weights based on the importance metric;
- and
- update the neural network model based on the plurality of quantized weights.
A non-transitory computer readable storage medium storing instructions which, when executed, cause a processing device to:
- determine, during training of a neural network model, for each of a plurality of training iterations, a corresponding value for a plurality of weights;
- calculate, for each weight of the plurality of weights and based on the corresponding value for each of the plurality of weights, an importance metric;
- calculate a plurality of quantized weights by binning the plurality of weights based on the importance metric;
- and
- update the neural network model based on the plurality of quantized weights.
Dependencies are inferred from the claim text.