Methods and processors for training a neural network

A system for using a Neural Network (NN), the NN comprising an input layer, an output layer, and a plurality of intermediate layers, the system comprising a controller and a memory storing a plurality of executable instructions which, when executed by the controller, cause the system to:

• during a first training iteration of the NN:
• determine a first continuous sequence of intermediate layers from the plurality of intermediate layers of the NN,
• the input layer, the first continuous sequence of intermediate layers, and the output layer forming a first sub-network of the NN;
• train the first sub-network based on training data;
• during a second training iteration of the NN:
• determine a second continuous sequence of intermediate layers from the plurality of intermediate layers of the NN,
• the input layer, the second continuous sequence of intermediate layers, and the output layer forming a second sub-network of the NN,
• the second continuous sequence of intermediate layers being different from the first sequence of continuous layers, the second continuous sequence of intermediate layers at least partially overlapping the first sequence of continuous layers;
• train the second sub-network based on the training data;
• during an inference iteration of the NN:
• select a target sub-network amongst the first sub-network and the second sub-network;
• and
• generate an inference output by employing only the target sub-network of the NN on inference data for reducing computational resources of the at least one processor for generating the inference output.