Cache-aware dynamic module selection

A processor-implemented method, comprising:

• generating at least one output, in a first inference round, using a first subset of modules of a computational model loaded in a cache memory from another memory;
• evaluating modules of the computational model to use for a second inference round, using a function that biases evaluation of the first subset of modules of the computational model already in the cache;
• and
• performing the second inference round with a second subset of modules of the computational module, based on the evaluation.

A processing system, comprising:

• means for generating at least one output, in a first inference round, using a first subset of modules of a computational model loaded in a cache memory from another memory;
• means for evaluating modules of the computational model to use for a second inference round, using a function that biases evaluation of the first subset of modules of the computational model already in the cache;
• and
• means for performing the second inference round with a second subset of modules of the computational module, based on the evaluation.