Inferencer graph for implementing machine learning model topology

A method for use with a computing system, the method comprising:

• at development time:
• receiving a machine learning model topology including a plurality of layers;
• and
• generating an internal representation graph of the machine learning model topology, wherein the internal representation graph includes a plurality of internal representation layers;
• and
• at least in part by performing one or more modifications to the internal representation graph, generating an inferencer graph including a plurality of inferencer layer blocks, wherein each inferencer layer block indicates an input buffer size, a logic function, and an output buffer size;
• and
• at deployment time, transmitting, to a plurality of processing devices, instructions to implement the machine learning model topology with the respective input buffer sizes, logic functions, and output buffer sizes indicated in the plurality of inferencer layer blocks of the inferencer graph.

A computing system comprising:

• a processor configured to:
• at development time:
• receive a machine learning model topology including a plurality of layers;
• receive a respective plurality of performance targets for the plurality of layers of the machine learning model topology;
• receive a plurality of processing device type indicators for the plurality of layers of the machine learning model topology;
• and
• based at least in part on the machine learning model topology, the plurality of performance targets, and the plurality of processing device indicators, generate an inferencer graph including a plurality of inferencer layer blocks, wherein each inferencer layer block indicates an input buffer size, a logic function, and an output buffer size;
• and
• at deployment time, transmit, to a plurality of processing devices that match the plurality of processing device type indicators, instructions to implement the machine learning model topology with the respective input buffer sizes, logic functions, and output buffer sizes indicated in the plurality of inferencer layer blocks of the inferencer graph, wherein each inferencer layer block of the inferencer graph meets the performance target for the corresponding layer of the machine learning model topology.