Mitigating loss of plasticity when training neural networks

One or more computer-readable storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations for reducing plasticity loss when training a neural network, the operations comprising:

• obtaining an original neural network including an output subnetwork, wherein the original neural network is configured to process a network input in accordance with trainable parameters of the original neural network to generate, from the output subnetwork, an original network output;
• training the original neural network during a first training phase, wherein the training comprises processing first training data items using the original neural network to generate original network outputs for the first training data items, and updating the trainable parameters of the original neural network using the original network outputs;
• creating first and second versions of the output subnetwork;
• forming an updated neural network comprising the original neural network and the first and second versions of the output subnetwork and configuring the updated neural network to process the network input to generate an updated network output from a combination of the original network output, a first network output from the first version of the output subnetwork, and a second network output from the second version of the output subnetwork;
• initializing trainable parameters of the first and second versions of the output subnetwork;
• freezing the trainable parameters of the second version of the output subnetwork after the initializing;
• and
• training the updated neural network during a second training phase, wherein the training comprises processing second training data items using the updated neural network to generate updated network outputs for the second training data items, and updating the trainable parameters of the updated neural network that are not frozen, using the updated network outputs.

A system comprising one or more computers and one or more storage devices storing instructions that when executed by one or more computers cause the one or more computers to perform operations for reducing plasticity loss when training a neural network, the operations comprising:

• obtaining an original neural network including an output subnetwork, wherein the original neural network is configured to process a network input in accordance with trainable parameters of the original neural network to generate, from the output subnetwork, an original network output;
• training the original neural network during a first training phase, wherein the training comprises processing first training data items using the original neural network to generate original network outputs for the first training data items, and updating the trainable parameters of the original neural network using the original network outputs;
• creating first and second versions of the output subnetwork;
• forming an updated neural network comprising the original neural network and the first and second versions of the output subnetwork and configuring the updated neural network to process the network input to generate an updated network output from a combination of the original network output, a first network output from the first version of the output subnetwork, and a second network output from the second version of the output subnetwork;
• initializing trainable parameters of the first and second versions of the output subnetwork;
• freezing the trainable parameters of the second version of the output subnetwork after the initializing;
• and
• training the updated neural network during a second training phase, wherein the training comprises processing second training data items using the updated neural network to generate updated network outputs for the second training data items, and updating the trainable parameters of the updated neural network that are not frozen, using the updated network outputs.