Minimum Deep Learning with Gating Multiplier

A computing system, comprising:

• one or more processors;
• and
• one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising:
• obtaining data descriptive of a neural network comprising one or more network units and one or more gating units, the one or more gating units comprising one or more gating paths associated with the one or more network units;
• training the neural network to learn one or more network parameters of the one or more network units and one or more gating parameters of the one or more gating units;
• sparsifying the neural network based at least in part on the one or more gating parameters of the one or more gating units to generate a sparsified neural network;
• and
• deploying the sparsified neural network to perform inference.

A computing system, comprising:

• one or more processors;
• and
• one or more non-transitory computer-readable media that collectively store:
• a sparsified neural network that was obtained by:
• obtaining data descriptive of a neural network comprising one or more network units and one or more gating units, the one or more gating units comprising one or more gating paths associated with the one or more network units;
• training the neural network to learn one or more network parameters of the one or more network units and one or more gating parameters of the one or more gating units;
• and
• sparsifying the neural network based at least in part on the one or more gating parameters of the one or more gating units to generate the sparsified neural network;
• and
• instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising:
• receiving an input;
• generating, based on the input, a prediction from the sparsified neural network;
• and
• returning an output based on the prediction.