Automated distribution of models for execution on a non-edge device and an edge device

A computer-implemented method comprising:

• receiving a request to generate an execution plan for executing a machine learning (ML) model using at least one of an edge device and a non-edge device, the request to include at least one objective for the execution of the ML model;
• generating the execution plan for the usage of the ML model based at least in part on the at least one objective, comparative execution information and network latency information, wherein the comparative execution information is based at least in part on executing the usage of the ML model on non-edge device hardware and executing or simulating on the non-edge device hardware execution of the usage of the ML model by hardware of the at least one edge device and comparing execution characteristics of executing the ML model on either the non-edge device or the edge device, and wherein the execution characteristics of executing the ML model on the edge device are further based at least in part on a hardware characteristic of the edge device;
• and
• executing at least one portion of the usage of the ML model on at least one of a host of the non-edge device and the edge device based at least in part on the execution plan.

A system comprising:

• an edge device;
• and
• a non-edge device coupled to the edge device, the non-edge device including a memory to store program code and a processor to execute the stored program code to cause the non-edge device to:
• receive a request to generate an execution plan for executing a ML model using one of the edge device and the non-edge device, the request to include at least one objective for the execution of the ML model;
• generate the execution plan for the usage of the ML model based at least in part on the at least one objective, comparative execution information and network latency information, wherein the comparative execution information is based at least in part on executing the usage of the ML model on non-edge device hardware and executing or simulating on the non-edge device hardware execution of the usage of the ML model by hardware of the at least one edge device and comparing execution characteristics of executing the ML model on either the non-edge device or the edge device, and wherein the execution characteristics of executing the ML model on the edge device are further based at least in part on a hardware characteristic of the edge device;
• and
• execute at least one portion of the usage of the ML model on a host of the non-edge device in conjunction with the edge device based at least in part on the execution plan.