Jointly Pruning and Quantizing a Neural Network

A computing system for pruning and quantizing a neural network, comprising:

• an instruction data store for storing computer-readable instructions;
• and
• a processing system for executing the computer-readable instructions in the data store, to perform operations including:
• receiving an original neural network having a structure with multiple levels, the original neural network having a first storage size;
• converting the original neural network into a quantized-preprocessed neural network by adding quantization logic to the original neural network;
• identifying an original set of groups of weight parameters used by the quantized-preprocessed neural network, each group in the original set of groups being associated with part of a structure of the quantized-preprocessed neural network;
• and
• performing preparatory training in which the quantized-preprocessed neural network is trained without pruning, to produce a pretrained neural network;
• jointly pruning and quantizing the pretrained neural network into a final neural network having a second storage size that is less than the first storage size, the jointly pruning and quantizing repeating operations of:
• identifying a prescribed number of groups of the original set of groups as redundant, a remainder of the original set of groups being to-be-retained groups,
• the redundant groups being groups in the original set of groups that are to be removed in the final neural network, and the to-be-retained groups being groups that are to be retained in the final neural network, and
• determining quantization parameters that govern quantization used in levels of pretrained neural network;
• and
• after the jointly pruning and quantizing, performing post-pruning training in which the to-be-retained groups are trained without performing further pruning,
• a target device being capable of storing and running the final neural network with fewer memory and processing resources than the original neural network.

A computer-readable storage medium for storing computer-readable instructions, a processing system executing the computer-readable instructions to perform operations, the operations comprising:

• receiving an original neural network having a structure with multiple levels, the original neural network having a first storage size;
• converting the original neural network into a quantized-preprocessed neural network by adding quantization logic to the original neural network,
• the converting including adding weight-quantization logic that simulates effects of converting weight parameters used by the original neural network into quantized weight parameters, and/or adding activation-quantization logic that simulates effects of converting activations produced by the layers of the original neural network into quantized activation information;
• identifying an original set of groups of weight parameters used by the quantized-preprocessed neural network, each group in the original set of groups being associated with part of a structure of the quantized-preprocessed neural network;
• and
• jointly pruning and quantizing the quantized-preprocessed neural network into a final neural network having a second storage size that is less than the first storage size, the jointly pruning and quantizing repeating operations of:
• identifying a prescribed number of groups of the original set of groups as redundant, a remainder of the original set of groups being to-be-retained groups,
• the redundant groups being groups in the original set of groups that are to be removed in the final neural network, and the to-be-retained groups being groups that are to be retained in the final neural network, and
• determining quantization parameters that govern quantization used in levels of the final neural network;
• and
• after the joint pruning and quantization, determining bit widths used by the levels of the quantized-preprocessed neural network based on the quantization parameters that have been determined.