Efficient execution of machine learning models in heterogeneous processing environments

A processing system, comprising:

• at least one memory having executable instructions stored thereon;
• and
• one or more processors configured to execute the executable instructions to cause the processing system to:
• partition a graph representing a machine learning model into a plurality of subgraphs, each subgraph representing a portion of the machine learning model;
• for each subgraph, simulate a plurality of execution paths based on permutations of using different processing unit types to execute portions of the subgraph and starting with each input source processing unit type selected from the different processing unit types;
• for each subgraph, select an execution path from the plurality of execution paths having a lowest cost;
• and
• implement the machine learning model based on the selected execution path for each subgraph.

A processing system, comprising:

• means for partitioning a graph representing a machine learning model into a plurality of subgraphs, each subgraph representing a portion of the machine learning model;
• means for simulating, for each subgraph, a plurality of execution paths based on permutations of using different processing unit types to execute portions of the subgraph and starting with each input source processing unit type selected from the different processing unit types;
• means for selecting, for each subgraph, an execution path from the plurality of execution paths having a lowest cost;
• and
• means for implementing the machine learning model based on the selected execution path for each subgraph.

A non-transitory computer-readable medium having executable instructions stored thereon which, when executed by one or more processors, perform an operation comprising:

• partitioning a graph representing a machine learning model into a plurality of subgraphs, each subgraph representing a portion of the machine learning model;
• for each subgraph, simulating a plurality of execution paths based on permutations of using different processing unit types to execute portions of the subgraph and starting with each input source processing unit type selected from the different processing unit types;
• for each subgraph, selecting an execution path from the plurality of execution paths having a lowest cost;
• and
• implementing the machine learning model based on the selected execution path for each subgraph.