Annealed dropout training of neural networks
AssigneeInternational Business Machines Corporation
InventorsVaibhava Goel, Steven John Rennie, Samuel Thomas, Ewout van den Berg
Life of this patent
Estimating the term…
Abstract
Systems and methods for training a neural network to optimize network performance, including sampling an applied dropout rate for one or more nodes of the network to evaluate a current generalization performance of one or more training models. An optimized annealing schedule may be generated based on the sampling, wherein the optimized annealing schedule includes an altered dropout rate configured to improve a generalization performance of the network. A number of nodes of the network may be adjusted in accordance with a dropout rate specified in the optimized annealing schedule. The steps may then be iterated until the generalization performance of the network is maximized.
Claims12
A method for training a neural network to optimize network performance of a network of interconnected computers, comprising:
- iteratively sampling an applied dropout rate for one or more nodes of the network to evaluate a current generalization performance of one or more training models;
- iteratively generating, using a processor, an optimized annealing schedule based on the sampling, wherein the optimized annealing schedule includes an altered dropout rate configured to improve a generalization performance of the network;
- increasing a realized capacity of the neural network by iteratively adjusting a number of nodes of the network in accordance with a dropout rate specified in the optimized annealing schedule until the generalization performance of the network is maximized.
Dependencies are inferred from the claim text.