Fixed, random, recurrent matrices for increased dimensionality in neural networks

A computer program product for operating a neural network, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor, so as to cause the processor to:

• train a neural network comprising successive layers, wherein:
• the successive layers include an input layer, a hidden layer, and an output layer;
• the input layer has n input nodes connected to output nodes of the output layer via the hidden layer;
• the hidden layer including m hidden nodes;
• and wherein the neural network is trained through connecting the n input nodes to a subset of k nodes of the m hidden nodes via respective synaptic connections to reduce number of learned parameters during the training, wherein each node in the subset of k nodes is associated with a training weight, which is used to form an n×k input matrix Win, wherein the training weights for the m hidden nodes are changeable during the training, and wherein a predetermined subset of m−k nodes of the hidden layer are not connected by any node of the input layer, wherein each node in the m−k nodes is associated with a fixed weight, wherein the fixed weights for the m−k nodes remain unchanged during the training;
• and the neural network is set as a feed-forward neural network, the feed-forward neural network including a backpropagation learning algorithm, wherein the backpropagation learning algorithm comprises running the neural network so as to train the training weights of the n×k input matrix Win;
• and
• couple input data into input nodes of the input layer.

A neural network, comprising a processor operably connected to a memory, the memory containing program instructions to, when executed on the processor:

• train a neural network comprising successive layers, wherein:
• the successive layers include an input layer, a hidden layer, and an output layer;
• the input layer has n input nodes connected to output nodes of the output layer via the hidden layer;
• the hidden layer including m hidden nodes;
• and wherein the neural network is trained through connecting the n input nodes to a subset of k nodes of the m hidden nodes via respective synaptic connections to reduce number of learned parameters during the training, wherein each node in the subset of k nodes is associated with a training weight, which is used to form an n×k input matrix Win, wherein the training weights for the m hidden nodes are changeable during the training, and wherein a predetermined subset of m−k nodes of the hidden layer are not connected by any node of the input layer, wherein each node in the m−k nodes is associated with a fixed weight, wherein the fixed weights for the m−k nodes remain unchanged during the training;
• and the neural network is set as a feed-forward neural network, the feed-forward neural network including a backpropagation learning algorithm, wherein the backpropagation learning algorithm comprises running the neural network so as to train the training weights of the n×k input matrix Win;
• and
• couple input data into input nodes of the input layer.