Reparameterization of selective networks for end-to-end training
OwnerMahmoud Salem, Frederick Tung +2inventors
InventorsMahmoud Salem, Frederick Tung, Mohamed O. Ahmed, Gabriel Oliveira
Life of this patent
Estimating the term…
Abstract
A method is provided for training a selective network that includes a selection node for selecting whether to make a prediction. During training, the selection node is reparameterized as a differentiable function of learnable parameters acting on noise from a base distribution. The differentiable function approximates a sampling from a categorical distribution.
Claims24
A method of training a selective network, wherein:
- the selective network includes a selection node for selecting whether to make a prediction;
- wherein:
- during training, the selection node is reparameterized as a differentiable function of learnable parameters acting on noise from a base distribution;
- wherein the differentiable function approximates a sampling from a categorical distribution.
A data processing system, comprising;
- at least one processor;
- a memory coupled to the at least one processor, the memory containing instructions which, when executed by the at least one processor, cause the at least one processor to:
- train a selective network, wherein the selective network includes a selection node for selecting whether to make a prediction;
- and
- during training, reparameterize the selection node as a differentiable function of learnable parameters acting on noise from a base distribution, wherein the differentiable function approximates a sampling from a categorical distribution.
A computer program product comprising a non-transitory tangible computer-readable medium having computer-readable instructions embodied therewith, wherein the instructions, when executed by at least one processor, cause the at least one processor to:
- train a selective network, wherein the selective network includes a selection node for selecting whether to make a prediction;
- and
- during training, reparameterize the selection node as a differentiable function of learnable parameters acting on noise from a base distribution, wherein the differentiable function approximates a sampling from a categorical distribution.
Dependencies are inferred from the claim text.