Method and apparatus for compressing neural network

An apparatus for compressing a neural network, the apparatus comprising:

• at least one processor;
• and
• a memory storing instructions, the instructions when executed by the at least one processor, cause the at least one processor to perform operations, the operations comprising:
• acquiring a to-be-compressed trained neural network;
• selecting at least one layer from layers of the neural network as a to-be-compressed layer;
• performing following processing steps sequentially on each of the to-be-compressed layers in descending order of a number of level of the to-be-compressed layer in the neural network:
• determining a pruning ratio based on a total number of parameters included in the to-be-compressed layer, selecting a parameter for pruning from the parameters included in the to-be-compressed layer based on the pruning ratio and a parameter value threshold, and training the pruned neural network based on a preset training sample using a machine learning method;
• and
• determining the neural network obtained after performing the processing steps on the selected at least one to-be-compressed layer as a compressed neural network, and storing the compressed neural network,
• wherein selecting a parameter for pruning from the parameters included in the to-be-compressed layer based on the pruning ratio and a parameter value threshold, and training the pruned neural network based on a preset training sample using a machine learning method comprises:
• performing following pruning training operations:
• determining a product of a specified ratio and the total number as a pruning number, and selecting, from the parameters included in the to-be-compressed layer, the pruning number of parameters having an absolute value of a parameter value not greater than the parameter value threshold for pruning in ascending order of the absolute value of the parameter value, wherein the specified ratio is not greater than the pruning ratio;
• training the pruned neural network based on the training sample using the machine learning method;
• and
• subtracting the ratio from the pruning ratio to obtain a first ratio, resetting the specified ratio to a positive number not greater than the first ratio, and continuing the pruning training operations in response to determining that the accuracy of the current trained neural network is not lower than the preset accuracy and the ratio is lower than the pruning ratio.

A non-transitory computer-readable storage medium storing a computer program, the computer program when executed by one or more processors, causes the one or more processors to perform operations, the operations comprising:

• acquiring a to-be-compressed trained neural network;
• selecting at least one layer from layers of the neural network as a to-be-compressed layer;
• performing following processing steps sequentially on each of the to-be-compressed layers in descending order of a number of level of the to-be-compressed layer in the neural network:
• determining a pruning ratio based on a total number of parameters included in the to-be-compressed layer, selecting a parameter for pruning from the parameters included in the to-be-compressed layer based on the pruning ratio and a parameter value threshold, and training the pruned neural network based on a preset training sample using a machine learning method;
• and
• determining the neural network obtained after performing the processing steps on the selected at least one to-be-compressed layer as a compressed neural network, and storing the compressed neural network,
• wherein selecting a parameter for pruning from the parameters included in the to-be-compressed layer based on the pruning ratio and a parameter value threshold, and training the pruned neural network based on a preset training sample using a machine learning method comprises:
• performing following pruning training operations:
• determining a product of a specified ratio and the total number as a pruning number, and selecting, from the parameters included in the to-be-compressed layer, the pruning number of parameters having an absolute value of a parameter value not greater than the parameter value threshold for pruning in ascending order of the absolute value of the parameter value, wherein the specified ratio is not greater than the pruning ratio;
• training the pruned neural network based on the training sample using the machine learning method;
• and
• subtracting the ratio from the pruning ratio to obtain a first ratio, resetting the specified ratio to a positive number not greater than the first ratio, and continuing the pruning training operations in response to determining that the accuracy of the current trained neural network is not lower than the preset accuracy and the ratio is lower than the pruning ratio.