Multi-iteration compression for deep neural networks
AssigneeBeijing Deephi Intelligent Technology Co., Ltd.
InventorsXin Li, Song Han, Shijie Sun, Yi Shan
Life of this patent
Estimating the term…
Abstract
A multi-iteration method for compressing a deep neural network into a sparse neural network without degrading the accuracy is disclosed herein. In an example, the method includes determining a respective initial compression ratio for each of a plurality of matrices characterizing the weights between the neurons of the neural network, compressing each of the plurality of matrices based on the respective initial compression ratio, so as to obtain a compressed neural network, and fine-tuning the compressed neural network.
Claims16
A method for compressing a neural network, wherein a plurality of weights between the neurons of said neural network are characterized by a plurality of matrices, said method comprising:
- determining a respective initial compression ratio for each of said plurality of matrices by:
- compressing each of the plurality of matrices according to a plurality of densities with each density representing a respective percentage of non-zero elements remaining after compression;
- measuring a Word Error Ratio (WER) of the neural network when the plurality of matrices are compressed under different densities;
- drawing a density-WER curve for each matrix of the plurality of matrices based on measured WERs as a function of different densities;
- determining a respective density for each matrix of the plurality of matrices, wherein the respective density is determined at an inflection point in a respective density-WER curve;
- selecting the respective density as the respective initial compression ratio;
- compressing each of said plurality of matrices based on said respective initial compression ratio, so as to obtain a compressed neural network;
- and fine-tuning said compressed neural network.
Dependencies are inferred from the claim text.