Back to results
US 10,832,135 B2Granted
Automatic thresholds for neural network pruning and retraining
AssigneeSamsung Electronics Co., Ltd.
InventorsZhengping Ji, John Wakefield Brothers, Ilia Ovsiannikov, Eunsoo Shim
Life of this patent
Estimating the term…
Abstract
An embodiment includes a method, comprising: pruning a layer of a neural network having multiple layers using a threshold; and repeating the pruning of the layer of the neural network using a different threshold until a pruning error of the pruned layer reaches a pruning error allowance.
Claims20
A method, comprising:
- pruning a layer of a neural network using a threshold, the neural network comprising multiple layers;
- and
- repeating pruning of the layer of the neural network using a different threshold for each iteration of the repeated pruning until a pruning error of the layer equals a pruning error allowance for the layer, the pruning error being determined as a quantity of a length of a vector of pruned weights of the layer for a current iteration minus a vector of weights of the layer for a previous iteration divided by a quantity of a total length of a vector of initial weights of the layer minus a length of the vector of pruned weights of the layer for the current iteration.
A method, comprising:
- pruning one or more times a layer of a neural network using a different threshold for each iteration of pruning until a pruning error of a currently pruned layer equals a pruning error allowance for the layer, the neural network comprising multiple layers, and the pruning error being determined as a quantity of a length of a vector of pruned weights of the layer for a current iteration minus a vector of weights of the layer for a previous iteration divided by a quantity of a total length of a vector of initial weight of the layer minus a length of the vector of pruned weights of the layer for the current iteration;
- and
- retraining the neural network using only weights remaining after pruning the one or more times.
A system, comprising:
- a memory;
- and
- a processor coupled to the memory, the processor:
- pruning a layer of a neural network using a threshold, the neural network comprising multiple layers;
- and
- repeating pruning of the layer of the neural network using a different threshold for each iteration of the repeated pruning until a pruning error of a currently pruned layer equals a pruning error allowance for the layer, the pruning error being determined as a quantity of a length of a vector of pruned weights of the layer for a current iteration minus a vector of weights of the layer for a previous iteration divided by a quantity of a total length of a vector of initial weights of the layer minus a length of the vector of pruned weights of the layer for the current iteration.
3 more elements
Dependencies are inferred from the claim text.