Artificial neural network and method of training an artificial neural network with epigenetic neurogenesis

A method for retraining an artificial neural network trained on data from an old task, the artificial neural network comprising an input layer having a plurality of input layer nodes, a plurality of hidden layers comprising at least a first hidden layer and a second hidden layer each having a plurality of hidden layer nodes, an output layer having a plurality of output layer nodes, and a plurality of old connections connecting the plurality of input layer nodes, the plurality of hidden layer nodes, and the output layer nodes, the method comprising:

• training the artificial neural network on data from the old task and data from a new task different than the old task;
• calculating a drift, utilizing Sliced Wasserstein Distance, in activation distributions of the plurality of hidden layer nodes during training of the artificial neural network with the data from the old task and data from the new task;
• calculating a number of additional nodes to add to at least one of the plurality of hidden layers based on the drift in the activation distributions;
• resetting connection weights between the plurality of input layer nodes, the plurality of hidden layer nodes, and the plurality of output layer nodes to values before the training of the artificial neural network on the data from the new task;
• adding a first set of additional nodes of the number of additional nodes to the first hidden layer of the plurality of hidden layers, connecting the first set of additional nodes with new connections, and not connecting the first set of additional nodes added to the first hidden layer to the plurality of hidden layer nodes in the second hidden layer;
• and
• training both the old connections and the new connections of the artificial neural network on data from the new task,
• wherein the calculating the number of additional nodes is calculated according to Equation 1:
• Nnodes=c*log(D)+b (Equation 1)
• wherein Nnodes is the number of additional nodes, c and b are user-specified constants and D is the drift in the activation distributions.