System and method for profiling on-chip performance of neural network execution

A method comprising:

• accessing a first static schedule of a target neural network for execution by a first processing device comprising a first set of resources, the target neural network comprising a set of layers defining a set of operations;
• generating a first set of expected performance metrics for the target neural network based on the first static schedule, the first set of expected performance metrics comprising a first expected performance metric for a first layer in the set of layers;
• capturing a first set of runtime performance metrics of the target neural network at a set of performance monitor registers in the processing device;
• associating a first runtime performance metric, in the first set of runtime performance metrics, with the first layer;
• in response to detecting a difference between the first runtime performance metric and the first expected performance metric exceeding a first threshold, identifying the first layer as a deviant layer in a subset of deviant layers in the set of layers;
• generating an interface indicating the subset of deviant layers;
• and
• serving the interface to a user.

A method comprising:

• accessing a static schedule of a target neural network for execution by a first processing device, the target neural network comprising a set of layers;
• generating a set of expected performance metrics of the target neural network based on the static schedule, the set of expected performance metrics comprising a first expected performance metric of a first operation in the set of operations;
• capturing a set of runtime performance metrics at a set of performance monitor registers during execution of the target neural network at the processing device, the set of runtime performance metrics comprising a first runtime performance metric of the first operation;
• calculating a difference between the first runtime performance metric and the first expected performance metric;
• in response to detecting the difference exceeding a threshold, identifying the first operation as a deviant operation;
• and
• serving a notification indicating the deviant operation to a user.