Computer-implemented methods and systems for DNN weight pruning for real-time execution on mobile devices
AssigneeNortheastern University
InventorsYanzhi Wang, Xiaolong Ma
Life of this patent
Estimating the term…
Abstract
A computer-implemented method is disclosed for compressing a deep neural network (DNN) model by DNN weight pruning to accelerate DNN inference on mobile devices. The method includes the steps of (a) performing an intra-convolution kernel pruning of the DNN model wherein a fixed number of weights are pruned in each convolution kernel of the DNN model to generate sparse convolution patterns; (b) performing inter-convolution kernel pruning of the DNN model to generate connectivity sparsity, wherein inter-convolution kernel pruning comprises cutting connections between given input and output channels of the DNN model to remove corresponding kernels; and (c) training the DNN model compressed in steps (a) and (b).
Claims18
A computer-implemented method for compressing a deep neural network (DNN) model by DNN weight pruning to accelerate DNN inference on mobile devices, the method comprising the steps of:
- (a) performing an intra-convolution kernel pruning of the DNN model wherein a fixed number of weights are pruned in each convolution kernel of the DNN model to generate sparse convolution patterns;
- (b) performing inter-convolution kernel pruning of the DNN model to generate connectivity sparsity, wherein inter-convolution kernel pruning comprises removing at least one kernel corresponding to a given input channel and a given output channel to cut connections between the given input channel and the given output channel of the DNN model to remove corresponding kernels, wherein said removing the at least one kernel comprises:
- retaining at least one kernel for each filter, and
- reducing a length of each filter;
- (c) training the DNN model compressed in steps (a) and (b);
- and
- (d) removing memory-level load redundancy corresponding to the DNN model trained in (c).
A computer system, comprising:
- at least one processor;
- memory associated with the at least one processor;
- and
- a program supported in the memory for compressing a deep neural network (DNN) model by DNN weight pruning to accelerate DNN inference on mobile devices, the program containing a plurality of instructions which, when executed by the at least one processor, cause the at least one processor to:
- (a) perform an intra-convolution kernel pruning of the DNN model wherein a fixed number of weights are pruned in each convolution kernel of the DNN model to generate sparse convolution patterns;
- (b) perform inter-convolution kernel pruning of the DNN model to generate connectivity sparsity, wherein inter-convolution kernel pruning comprises removing at least one kernel corresponding to a given input channel and a given output channel to cut connections between the given input channel and the given output channel of the DNN model to remove corresponding kernels, wherein said removing the at least one kernel comprises:
- retaining at least one kernel for each filter, and
- reducing a length of each filter;
- (c) train the DNN model compressed in steps (a) and (b);
- and
- (d) removing memory-level load redundancy corresponding to the DNN model trained in (c).
A computer program product for compressing a deep neural network (DNN) model by DNN weight pruning to accelerate DNN inference on mobile devices, said computer program product residing on a non-transitory computer readable medium having a plurality of instructions stored thereon which, when executed by a computer processor, cause that computer processor to:
- (a) perform an intra-convolution kernel pruning of the DNN model wherein a fixed number of weights are pruned in each convolution kernel of the DNN model to generate sparse convolution patterns;
- (b) perform inter-convolution kernel pruning of the DNN model to generate connectivity sparsity, wherein inter-convolution kernel pruning comprises removing at least one kernel corresponding to a given input channel and a given output channel to cut connections between the given input channel and the given output channel of the DNN model to remove corresponding kernels, wherein said removing the at least one kernel comprises:
- retaining at least one kernel for each filter, and
- reducing a length of each filter;
- (c) train the DNN model compressed in steps (a) and (b);
- and
- (d) removing memory-level load redundancy corresponding to the DNN model trained in (c).
Dependencies are inferred from the claim text.