Back to results
US 12,008,474 B2Granted
Automatic thresholds for neural network pruning and retraining
AssigneeSamsung Electronics Co., Ltd.
InventorsZhengping Ji, John Wakefield Brothers, Ilia Ovsiannikov, Eunsoo Shim
Life of this patent
Estimating the term…
Abstract
An embodiment includes a method, comprising: pruning a layer of a neural network having multiple layers using a threshold; and repeating the pruning of the layer of the neural network using a different threshold until a pruning error of the pruned layer reaches a pruning error allowance.
Claims20
A method, comprising:
- pruning, by a processor, a layer of a neural network using a first threshold, the neural network comprising multiple layers, the pruning comprising configuring weights of the layer based on the first threshold;
- determining, by the processor, a pruning error of the layer based on a percentage of the configured weights of the layer with respect to an initial number of weights of the layer;
- and
- repeatedly configuring, by the processor, the weights of the layer until the pruning error of the layer equals a pruning error allowance for the layer, wherein the repeatedly configuring the weights of the layer allows a percentage of the number of weights remaining after the repeatedly configuring the weights of the layer with respect to the initial number of weights of the layer to be determined, and wherein each iteration of the repeatedly configuring the weights of the layer uses a different first threshold based on the pruning error.
1 more element
A method, comprising:
- pruning a layer of a neural network, by a processor, one or more times using a different first threshold for each iteration of pruning, the pruning comprising configuring weights of the layer based on the different first threshold until a pruning error of the equals a pruning error allowance for the layer, the neural network comprising multiple layers, the pruning error of being based on a percentage of the configured weights of the layer with respect to an initial number of weights of the layer, and wherein the repeatedly configuring the weights of the layer allows a percentage of the number of weights remaining after the configuring the weights of the layer with respect to the initial number of weights of the layer to be determined;
- and
- retraining, by the processor, the neural network using only weights remaining after one or more prunings.
A system, comprising:
- a memory;
- and
- a processor coupled to the memory, the processor configured to:
- prune a layer of a neural network using a first threshold, the neural network comprising multiple layers, the pruning comprising configuring weights of the layer based on the first threshold;
- determine a pruning error of the layer that is based on a percentage of the configured weights of the layer with respect to an initial number of weights of the layer;
- and
- repeatedly configure the weights of the layer until the pruning error of the layer equals a pruning error allowance for the layer, wherein the repeatedly configuring the weights of the layer allows a percentage of the number of weights remaining after the repeatedly configuring the weights of the layer with respect to the initial number of weights of the layer to be determined, and wherein each iteration of the repeatedly configuring the weights of the layer uses a different first threshold based on the pruning error.
4 more elements
Dependencies are inferred from the claim text.