Neural network selection

A system for identifying a neural network for processing data, the system comprising a first processing system comprising one or more processors configured to carry out a method comprising:

• clustering a training dataset into a plurality of data clusters based on similarities in activation patterns generated in neurons of a teacher neural network in response to inputting the training dataset into the teacher neural network;
• training a student neural network for processing each of the plurality of data clusters;
• and
• providing a data classifier neural network for identifying one or more of the trained student neural networks to process data based on a data cluster of the data.

A computer-implemented method of identifying, from a plurality of student neural networks trained by clustering a training dataset into a plurality of data clusters based on similarities in activation patterns generated in neurons of a teacher neural network in response to inputting the training dataset into the teacher neural network, and by, for each data cluster, inputting data from the data cluster into a student neural network and into the teacher neural network, and optimising parameters of the student neural network based on the output of the teacher neural network;

• a student neural network for processing data using one or more processors of a second processing system based on a data cluster of the data identified by a data classifier neural network trained by inputting the training dataset into the data classifier neural network, and training the data classifier neural network to identify each data cluster in the training dataset based on the corresponding data cluster provided by the clustering;
• the method comprising:
• receiving data, using the one or more processors of the second processing system;
• inputting, using the one or more processors of the second processing system, the received data into the data classifier neural network;
• classifying, using the one or more processors of the second processing system, the received data as belonging to one or more of the plurality of data clusters using the data classifier neural network;
• recording, using the one or more processors of the second processing system, a history of the data clusters classified by the data classifier neural network;
• and
• identifying, using the one or more processors of the second processing system, one or more of the trained student neural networks for processing further data received by the one or more processors of the second processing system, based on the recorded history of the data clusters classified by the data classifier neural network.