Removing nodes from machine-trained network based on introduction of probabilistic noise during training

A non-transitory machine-readable medium storing a program which when executed by at least one processing unit reduces complexity of a machine-trained (MT) network that receives input data and computes output data for each input data, the MT network comprising a plurality of computation nodes arranged in a plurality of layers with each layer of a subset of the layers comprising a plurality of channels of computation nodes, wherein each computation node of at least a subset of the layers (i) generates a node output value and (ii) uses node output values of other computation nodes in one or more previous layers as node input values, the program comprising sets of instructions for:

• during training of the MT network, introducing probabilistic noise to the node output values of a set of the computation nodes;
• determining a subset of the computation nodes for which the introduction of the probabilistic noise to the node output value does not affect the computed output data for the network by using a loss function term that estimates information transmitted by each of the layers, the loss function term comprising a summation over terms for each layer of the subset of layers, the term for each layer in the subset of layers comprising a summation over loss terms for each of the channels of the layer, the term for a particular one of the layers comprising a summation over sigmoid functions of a variance of the probabilistic noise introduced for each channel of the particular layer, wherein the term for each layer in the subset of layers penalizes the layer based on a number of remaining channels in the layer;
• and
• removing the subset of computation nodes from the trained MT network.