Hierarchical partitioning of operators

A method, comprising:

• executing a machine learning framework to generate operators of a neural network, wherein at least some of the neural network operators are executable on an acceleration engine;
• identifying neural network operators from the neural network that are supported by a compiler for an acceleration engine;
• partitioning, by a partitioner, the neural network operators that are supported by the compiler into a set of first operators that is initially determined to be executable by the acceleration engine and a set of second operators that is executable by a host processor;
• evaluating parameters of the first set of operators;
• determining that specific instances of neural network operators of the first set of operators include a parameter that is not supported by the acceleration engine;
• partitioning, by the partitioner, the operator that includes the parameter that is not supported by the acceleration engine for execution on the host processor;
• compiling a first model of a first subset of the operators supported by the compiler for execution on the acceleration engine;
• and
• compiling a second model of a second subset of the operators supported by the compiler for execution on the host processor, the second subset of the operators including the specific instances of neural network operators having the parameters that are not supported by the acceleration engine.

A system, comprising:

• a set of processors;
• and
• an acceleration engine,
• wherein the set of processors is configured to:
• execute a machine learning framework to generate neural network operators, wherein at least some of the operators are executable on the acceleration engine;
• partition, by a partitioner, the neural network operators into a first subset of operators that are initially determined to be supported by a compiler for the acceleration engine and a set of second operators that is executable by the set of processors;
• evaluate parameters of the first subset of operators;
• determine that a parameter of an operator of the first subset of operators initially determined to be supported by a compiler for the acceleration engine is not supported by the acceleration engine;
• and
• partition, by the partitioner, the operator of the first subset of operators initially determined to be capable of execution supported by the compiler for the acceleration engine for execution on the set of processors;
• compile a first model of the first subset of the operators supported by the compiler for execution on the acceleration engine;
• and
• compile a second model of a second subset of the operators supported by the compiler for execution on the set of processors, the second subset of the operators including specific instances of neural network operators having the parameters that are not supported by the acceleration engine.

A non-transitory computer readable medium having stored therein instructions for making one or more processors execute a method for partitioning operators of a neural network, the processor executable instructions comprising instructions for performing operations including:

• executing a machine learning framework to generate operators of a neural network, wherein at least some of the neural network operators are executable on an acceleration engine;
• identifying neural network operators from the neural network that are supported by a compiler for an acceleration engine;
• partitioning, by a partitioner, the neural network operators that are supported by the compiler into a set of first operators that is initially determined to be executable by the acceleration engine and a set of second operators that is executable by a host processor;
• evaluating parameters of the first set of operators;
• determining that specific instances of neural network operators of the first set of operators include a parameter that is not supported by the acceleration engine;
• partitioning, by the partitioner, the operator for execution on the host processor;
• compiling a first model of a first subset of the operators supported by the compiler for execution on the acceleration engine;
• and
• compiling a second model of a second subset of the operators supported by the compiler for execution on the host processor, the second subset of the operators including the specific instances of neural network operators having the parameters that are not supported by the acceleration engine.