US 2021/0264278 A1 · NEURAL NETWORK ARCHITECTURE PRUNING

Abstract

The disclosure describes one or more implementations of a neural network architecture pruning system that automatically and progressively prunes neural networks. For instance, the neural network architecture pruning system can automatically reduce the size of an untrained or previously-trained neural network without reducing the accuracy of the neural network. For example, the neural network architecture pruning system jointly trains portions of a neural network while progressively pruning redundant subsets of the neural network at each training iteration. In many instances, the neural network architecture pruning system increases the accuracy of the neural network by progressively removing excess or redundant portions (e.g., channels or layers) of the neural network. Further, by removing portions of a neural network, the neural network architecture pruning system can increase the efficiency of the neural network.

Claims20

Dependent claims

A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause a computing device to:

initialize a convolutional neural network comprising a plurality of batch-normalization layers and a plurality of network weights;
and
prune the convolutional neural network based on a pruning parameter while jointly learning the neural network weights and scaling parameters by iteratively:
updating the neural network weights and a scaling parameter for each portion;
determining portions of the convolutional neural network that generate scaling parameters not satisfying the pruning parameter;
and
modifying an architecture of the convolutional neural network by removing the determined portions.

4 more elements

A system comprising:

one or more memory devices comprising:
a convolutional neural network comprising a plurality of batch-normalization layers;
and
a plurality of network weights;
one or more computing devices configured to cause the system to:
prune the batch-normalization layers of the convolutional neural network while jointly learning the neural network weights and scaling parameters by iteratively performing, until convergence of the convolution neural network, by:
updating the neural network weights and scaling parameters of the plurality of batch-normalization layers;
identifying batch-normalization layers of the plurality of batch-normalization layers that generate scaling parameters that are below a pruning parameter;
and
modifying an architecture of the convolutional neural network by removing the batch-normalization layers from the convolutional neural network that generate scaling parameters that are below the pruning parameter.

7 more elements

Dependencies are inferred from the claim text.

Neural network architecture pruning

Life of this patent

Neural network architecture pruning

Life of this patent

Abstract

Claims20