Interpretable neural network

Computing apparatus comprising one or more processors and storage storing code arranged to run on the one or more processors, wherein the code is configured so as when run to perform operations of operating a neural network, wherein the neural network comprises a plurality of layers, the plurality of layers comprising:

• i) an input layer comprising a plurality of input nodes each configured to receive a respective one of a plurality of input elements, ii) one or more hidden layers each comprising a plurality of hidden nodes, each hidden node configured to receive sets of input parameters where each set describes an input probability distribution from one of the nodes in a previous layer of the network, and to output a set of output parameters describing an output probability distribution to a next layer of the network, and iii) an output layer comprising one or more output nodes each configured to output a respective output element, wherein the one or more hidden layers connect the input layer to the output layer;
• and wherein said operations comprise:
• at each of the input nodes, weighting the respective one of the plurality of input elements received by that input node by applying an instance of a first class of probability distribution to that input element, thereby generating a respective set of output parameters describing an output probability distribution;
• and
• from each of the input nodes, outputting the respective set of output parameters as input parameters to one or more nodes in a next, hidden layer of the network, and thereby propagating the respective set of output parameters through the one or more hidden layers to the output layer;
• said propagating comprising, at each of one or more nodes of at least one of the hidden layers, combining the sets of input parameters and weighting the combination by applying an instance of a second class of probability distribution to that combined set of input parameters, thereby generating a respective set of output parameters describing an output probability distribution for outputting to a next layer of the network, and wherein the first class of probability distribution is more sparsity inducing than the second class of probability distribution.

A computer program embodied on computer-readable storage and configured so as when run one or more processors to perform operations of operating a neural network, wherein the neural network comprises a plurality of layers, the plurality of layers comprising:

• i) an input layer comprising a plurality of input nodes each configured to receive a respective one of a plurality of input elements, ii) one or more hidden layers each comprising a plurality of hidden nodes, each hidden node configured to receive sets of input parameters where each set describes an input probability distribution from one of the nodes in a previous layer of the network, and to output a set of output parameters describing an output probability distribution to a next layer of the network, and iii) an output layer comprising one or more output nodes each configured to output a respective output element, wherein the one or more hidden layers connect the input layer to the output layer;
• and wherein said operations comprise:
• at each of the input nodes, weighting the respective one of the plurality of input elements received by that input node by applying an instance of a first class of probability distribution to that input element, thereby generating a respective set of output parameters describing an output probability distribution;
• and
• from each of the input nodes, outputting the respective set of output parameters as input parameters to one or more nodes in a next, hidden layer of the network, and thereby propagating the respective set of output parameters through the one or more hidden layers to the output layer;
• said propagating comprising, at each of one or more nodes of at least one of the hidden layers, combining the sets of input parameters and weighting the combination by applying an instance of a second class of probability distribution to that combined set of input parameters, thereby generating a respective set of output parameters describing an output probability distribution for outputting to a next layer of the network, and wherein the first class of probability distribution is more sparsity inducing than the second class of probability distribution.