Neural network compression

A non-transitory machine-readable storage medium with instructions stored thereon, wherein the instructions are executable by a machine to cause the machine to:

• access data comprising a definition of a neural network model, wherein the neural network model comprises a plurality of layers, and each of the plurality of layers comprises a respective set of weights;
• access training data for use in training of the neural network model, wherein the training comprises a plurality of training iterations;
• determine a subset of the weights to remove from a particular one of the plurality of layers;
• prune the weights of the particular layer to remove the subset of weights from the particular layer during a forward pass of a particular one of the training iterations, wherein removing the subset of weights results in a sparsified subset of weights remaining in the particular layer;
• quantize values of at least a portion of the weights of the particular layer during the forward pass of the particular training iteration, wherein after the forward pass the particular layer comprises a quantized-sparsified subset of weights;
• and
• generate, from the training, a compressed version of the neural network model based on the quantized-sparsified subset of weights.

A method comprising:

• performing a plurality of a training iterations using a set of training data to train a neural network model, wherein each of the plurality of training iterations comprises a respective forward pass and a respective backward pass, and the neural network model comprises a plurality of layers;
• in each one of the plurality of training iterations:
• determining a respective subset of weights of a particular one of the plurality of layer to remove;
• pruning the particular layer to remove the subset of weights in the forward pass of the corresponding training iteration, wherein weights of the particular layer remaining after removal of the subset of weights comprise sparsified weights of the particular layer;
• quantizing values of the sparsified weights of the particular layer from a higher precision to a lower precision to form a respective sparsified-quantized set of weights for the particular layer;
• determining activation values for the particular layer based on the sparsified-quantized set of weights;
• and
• performing backpropagation using the activation values during the corresponding backward pass;
• and
• generating a compressed version of the neural network model based on the training.

A system comprising:

• a data processor;
• a memory;
• and
• a neural network compression engine executable by the data processor to perform training of a neural network model, wherein the training of the neural network model generates a compressed version of the neural network model and comprises:
• a plurality of training iterations, wherein each of the plurality of training iterations comprises a forward pass and a backward pass;
• weight sparsification to prune weights of at least one particular layer of the neural network model, wherein the weight sparsification removes a subset of the weights based on a threshold value for weights in the particular layer, and the weight sparsification is performed in a forward pass of a particular one of the plurality of training iterations;
• and
• quantization of weights of the particular layer, wherein the weight sparsification and quantization of weights generate a quantized-sparsified set of weights for the particular layer, wherein the quantization of weights is also performed in the forward pass of the particular training iteration.