Freeze-out as a regularizer in training neural networks
AssigneeGE Precision Healthcare LLC
InventorsTao Tan, Min Zhang, Gopal Biligeri Avinash, Lehel Ferenczi, Levente Imre Török, Pál Tegzes
Life of this patent
Estimating the term…
Abstract
Systems and techniques that facilitate freeze-out as a regularizer in training neural networks are presented. A system can include a memory and a processor that executes computer executable components. The computer executable components can include: an assessment component that identifies units of a neural network, a selection component that selects a subset of units of the neural network, and a freeze-out component that freezes the selected subset of units of the neural network so that weights of output connections from the frozen subset of units will not be updated for a training run.
Claims20
A system, comprising:
- a memory that stores computer executable components;
- and
- a processor that executes computer executable components stored in the memory, wherein the computer executable components comprise:
- an assessment component that identifies units of a neural network;
- a selection component that selects a random subset of units of the neural network;
- and
- a freeze-out component that freezes the selected subset of units of the neural network so that weights of output connections from the frozen subset of units will not be updated for a training run;
- wherein the freezing out of the selected subset of units of the neural network is performed in lieu of dropping out the selected units such that architecture of the neural network remains unchanged.
A method, comprising:
- identifying, by a system comprising a processor, units of a neural network;
- selecting, by the system, a random subset of units of the neural network;
- and
- freezing, by the system, the selected subset of units of the neural network so that weights of output connections from the frozen subset of units will not be updated for a training run;
- wherein the freezing out of the selected subset of units of the neural network is performed in lieu of dropping out the selected units such that architecture of the neural network remains unchanged.
A computer readable storage device comprising instructions that, in response to execution, cause a system comprising a processor to perform operations, comprising:
- identifying, by the processor, units of a neural network;
- selecting, by the processor, a random subset of units of the neural network;
- and
- freezing, by the processor, the selected subset of units of the neural network so that weights of output connections from the frozen subset of units will not be updated for a training run;
- wherein the freezing out of the selected subset of units of the neural network is performed in lieu of dropping out the selected units such that architecture of the neural network remains unchanged.
Dependencies are inferred from the claim text.