Annotation override determination for a neural network

A method comprising:

• at runtime corresponding to execution of a neural network model on a system, for each respective layer of a plurality of layers of the neural network model:
• determining an operation performed at a node of a the respective layer of the neural network model;
• determining whether to override an annotation of the operation performed at the node based at least in part on a current condition of the system, the annotation having been assigned prior to runtime based at least in part on a cost of transferring the operation between a first and second processor or a number of cycles for completing the operation on at least one of the first or second processor, wherein the first processor has different hardware capabilities than the second processor, and the annotation indicates the first or second processor for performing the operation;
• and
• cause the operation to be executed in accordance with the determining whether to override the annotation.

A non-transitory computer-readable medium comprising instructions, which when executed by a computing device, cause the computing device to perform operations comprising:

• at runtime corresponding to execution of a neural network model, for each respective layer of a plurality of layers of the neural network model:
• determining an operation performed at a node of the respective layer of the neural network model;
• determining whether to override an annotation of the operation performed at the node based at least in part on a current condition of the computing device, the annotation indicating whether the operation is suggested to be performed on a first processor or a second processor having different hardware capabilities than the first processor, wherein the annotation having been assigned based at least in part on a cost of performing the operation at the first or second processor or a cost of transferring the operation between the first and second processor;
• and
• causing the operation to be executed in accordance with the determining whether to override the annotation.