Method and device for on-device learning based on multiple instances of inference workloads

A method of training a neural network using a circuit comprising a memory and a processing device, the method comprising:

• performing a first forward inference pass through the neural network based on input features to generate first activations, and generating an error based on a target value, and storing the error to the memory;
• and
• performing, for each layer of the neural network:
• a modulated forward inference pass based on the error to generate one or more modulated activations, and storing the one or more modulated activations to the memory;
• before, during or after the modulated forward inference pass, a second forward inference pass based on the input features to regenerate one or more of the first activations, and storing the one or more regenerated first activations to the memory;
• and
• updating one or more weights in the neural network based on the modulated activations and the one or more regenerated first activations.