Targeted incremental growth with continual learning in deep neural networks

A computer system for training a neural network, wherein the neural network comprises an input layer, an output layer, and one or more hidden layers between the input layer and the output layer, and wherein each layer comprises one or more nodes, such that the neural network comprises at least a first node that is not on the input layer and a second node that is not on the input layer, the computer system comprising:

• one or more processing cores;
• and
• computer memory in communication with the one or more processing cores, wherein the computer memory stores computer instructions that when executed by the one or more processing cores, cause the one or more processing cores to:
• iteratively train, through machine learning, the neural network on a training data set, wherein the computer memory stores instructions that, when executed by the one or more processing cores, cause the one or more processing cores to iteratively train the neural network by imposing a first is-not-equal-to regularization link between the first node and the second node, wherein the computer memory stores instructions that, when executed by the one or more processing cores, cause the one or more processing cores to impose the first is-not-equal-to regularization link between the first and second nodes by adding, during back-propagation of partial derivatives through the neural network for a datum in the training data set, a first regularization cost to a network error loss function for the first node that is inversely proportional to a difference between an activation value for the first node for the datum and an activation value for the second node for the datum.