Back to results
US 11,907,843 B2Granted
Importance-aware model pruning and re-training for efficient convolutional neural networks
AssigneeIntel Corporation
InventorsAnbang Yao, Yiwen Guo, Yurong Chen
Life of this patent
Estimating the term…
Abstract
Systems, apparatuses and methods may provide for conducting an importance measurement of a plurality of parameters in a trained neural network and setting a subset of the plurality of parameters to zero based on the importance measurement. Additionally, the pruned neural network may be re-trained. In one example, conducting the importance measurement includes comparing two or more parameter values that contain covariance matrix information.
Claims25
An apparatus comprising:
- a memory;
- and
- hardware logic coupled to the memory, wherein the hardware logic is implemented in one or more of configurable logic or fixed-functionality logic hardware, the hardware logic coupled to the memory to:
- conduct an importance measurement of a plurality of parametric generators in a trained neural network with one or more comparators that compare two or more parameter values that contain covariance matrix information, wherein the covariance matrix information is to indicate that first values of an inverse covariance matrix are greater than second values of the inverse covariance matrix, wherein the first values correspond to a first subset of a plurality of parameters and the second values correspond to a second subset of the plurality of parameters;
- set the first subset of the plurality of parameters to zero and bypass a modification of the second subset of the plurality of parameters to zero based on the first values being greater than the second values to obtain a pruned neural network, wherein one or more parameters in the first subset is to be less than one or more of the second subset of the plurality of parameters;
- re-train the pruned neural network;
- and
- iteratively conduct the importance measurement, wherein the hardware logic coupled to the memory is to iteratively set the first subset of the plurality of parameters to zero and iteratively re-train the pruned neural network until the pruned neural network satisfies a sparsity condition.
5 more elements
An apparatus comprising:
- a processor;
- and
- a memory including a set of instructions, which when executed by the processor, causes the apparatus to:
- conduct an importance measurement of a plurality of parameters in a trained neural network based on covariance matrix information, wherein the covariance matrix information is to indicate that first values of an inverse covariance matrix are greater than second values of the inverse covariance matrix, wherein the first values correspond to a first subset of the plurality of parameters and the second values correspond to a second subset of the plurality of parameters;
- set the first subset of the plurality of parameters to zero and bypass a modification of the second subset of the plurality of parameters to zero based on the first values being greater than the second values to obtain a pruned neural network;
- and
- re-train the pruned neural network.
4 more elements
A method comprising:
- conducting an importance measurement of a plurality of parameters in a trained neural network based on covariance matrix information, wherein the covariance matrix information is to indicate that first values of an inverse covariance matrix are greater than second values of the inverse covariance matrix, wherein the first values correspond to a first subset of the plurality of parameters and the second values correspond to a second subset of the plurality of parameters;
- setting the first subset of the plurality of parameters to zero and bypassing a modification of the second subset of the plurality of parameters to zero based on the first values being greater than the second values to obtain a pruned neural network;
- and
- re-training the pruned neural network.
1 more element
At least one non-transitory computer readable storage medium comprising a set of instructions, which when executed by a computing system, cause the computing system to:
- conduct an importance measurement of a plurality of parameters in a trained neural network based on covariance matrix information, wherein the covariance matrix information is to indicate that first values of an inverse covariance matrix are greater than second values of the inverse covariance matrix, wherein the first values correspond to a first subset of the plurality of parameters and the second values correspond to a second subset of the plurality of parameters;
- set the first subset of the plurality of parameters to zero and bypass a modification of the second subset of the plurality of parameters to zero based on the first values being greater than the second values to obtain a pruned neural network;
- and
- re-train the pruned neural network.
1 more element
Dependencies are inferred from the claim text.