Distributed machine learning compiler optimization

A method for optimizing compilation of one or more machine learning models, wherein said machine learning model is to be executed on one or more target edge devices, the method for a plurality of computing nodes and comprising:

• receiving instructions at each of the plurality of computing nodes to participate in a compiler optimization process for a compiler of said machine learning model, said machine learning model being generated in a machine learning framework and having a compute graph representation forming machine learning model information, wherein nodes of said compute graph representation are kernel operators necessary to execute the machine learning model and edges of said compute graph representation connect said kernel operators to define precedence constraints amongst the kernel operators;
• receiving instructions at each of the plurality of computing nodes to execute a round of optimization for the compiler optimization process including at least one respective sequencing and scheduling solution that was determined based on hardware information for the one or more target edge devices and the precedence constraints,
• applying at each computing node said at least one respective sequencing and scheduling solution, wherein each applied sequencing and scheduling solution maps the kernel operators to hardware resources and corresponds to a respective machine code which upon execution on the computing nodes results in a respective performance metric for the machine learning model;
• sending said performance metric from each of the compute nodes to the controller for identifying an optimal sequencing and scheduling solution having the best performance metric amongst the applied sequencing and scheduling solutions, and for implementing in the compiler the optimal sequencing and scheduling solution for executing the machine learning model on the one or more target edge devices.

A controller for optimizing compilation of one or more machine learning models, wherein said machine learning model is to be executed on one or more target edge devices, said controller in communication with a plurality of compute nodes and comprising:

• a processor configured to
• allocate compute nodes of said plurality of compute nodes to a compiler optimization process for a compiler of said machine learning model, said machine learning model being generated in a machine learning framework and having a compute graph representation forming machine learning model information, wherein nodes of said compute graph representation are kernel operators necessary to execute the machine learning model and edges of said compute graph representation connect said kernel operators to define precedence constraints amongst the kernel operators;
• schedule a round of optimization for the compiler optimization process amongst the allocated compute nodes, said compiler optimization process comprises applying at each allocated compute node at least one sequencing and scheduling solution that was determined based on hardware information for the one or more target edge devices and the precedence constraints, wherein each sequencing and scheduling solution maps the kernel operators upon hardware resources and upon compiling leads to execution on one of the compute nodes and results in a performance metric for the machine learning model;
• receive from the compute nodes said performance metric for said round of optimization;
• identify one or more sequencing and scheduling solutions having the best performance metric amongst the sequencing and scheduling solutions applied at the compute nodes;
• and
• implement in the compiler the optimal sequencing and scheduling solution for generating machine code to execute the machine learning model on the one or more target edge devices.

A system of plurality of compute nodes for optimizing compilation of one or more machine learning models, wherein said machine learning model is to be executed on one or more target edge devices, said plurality of compute nodes in communication with a controller and each node comprising:

• a processor configured to
• receive instructions to participate in a compiler optimization process for a compiler of said machine learning model, said machine learning model being generated in a machine learning framework and having a compute graph representation forming machine learning model information, wherein nodes of said compute graph representation are kernel operators necessary to execute the machine learning model and edges of said compute graph representation connect said kernel operators to define precedence constraints amongst the kernel operators;
• receive instructions to execute a round of optimization for the compiler optimization process including at least one respective sequencing and scheduling solution that was determined based on hardware information for the one or more target edge devices and the precedence constraints,
• applying said at least one respective sequencing and scheduling solution, wherein each applied sequencing and scheduling solution maps the kernel operators to hardware resources and corresponds to a respective machine code which upon execution on the computing nodes results in a respective performance metric for the machine learning model;
• send said performance metric to the controller for identifying an optimal sequencing and scheduling solution having the best performance metric amongst the applied sequencing and scheduling solutions, and for implementing in the compiler the optimal sequencing and scheduling solution for executing the machine learning model on the one or more target edge devices.