Allocating resources for a machine learning model

A computer-implemented method for allocating resources for machine learning models to run together on a special purpose machine learning model processor, the method comprising:

• generating, from machine learning models, computational dataflow graphs, each graph representing one of the machine learning models;
• scheduling and compiling, using a deterministic instruction set architecture that specifies functionality of a special purpose machine learning model processor, the computational dataflow graphs representing the machine learning models into executable binaries;
• determining an amount of resources required to execute each executable binary;
• determining an amount of resources available on the special purpose machine learning model processor;
• determining a first executable binary of a computational dataflow graph representing a machine learning model to execute on the special purpose machine learning model processor;
• determining a second executable binary of a computational dataflow graph representing a second machine learning model to execute on the special purpose machine learning model processor;
• and
• allocating resources for the first and second machine executable binaries to execute together on the special purpose machine learning model processor.

A system comprising one or more computers and one or more storage devices storing instructions that are executable by the one or more computers to cause the one or more computers to perform the operations of:

• receiving a machine learning model to execute on a special purpose machine learning model processor;
• generating, from the machine learning model, a computational dataflow graph representing the machine learning model comprising a plurality of nodes, a plurality of connector directed edges, and a plurality of parameter directed edges, wherein each node represents a respective operation performed by the machine learning model as part of the model output from the model input, wherein each connector directed edge connects a respective first node to a respective second node that represents an operation that receives, as input, an output of an operation represented by the respective first node, and wherein each parameter directed edge connects into a respective node and represents a flow of one or more parameters of the model as input to the operation represented by the respective node;
• scheduling operations of the computational dataflow graph;
• compiling the computational dataflow graph using a deterministic instruction set architecture that specifies functionality of a special purpose machine learning model processor;
• determining an amount of resources required to execute the computational dataflow graph representing the machine learning model;
• and
• allocating resources based on the determined amounts of resources required to execute the machine learning model represented by the computational dataflow graph.

A computer-implemented method for allocating resources for machine learning models to run together on a special purpose machine learning model processor, the method comprising:

• generating, from machine learning models, computational dataflow graphs, each graph representing one of the machine learning models;
• scheduling and compiling, using a deterministic instruction set architecture that specifies functionality of a special purpose machine learning model processor, the computational dataflow graphs representing the machine learning models into executable binaries;
• determining an amount of resources required to execute each executable binary;
• determining an amount of resources available on the special purpose machine learning model processor;
• determining a first executable binary of a computational dataflow graph representing a machine learning model to execute on the special purpose machine learning model processor;
• determine a second executable binary of a computational dataflow graph representing a second machine learning model to execute on the special purpose machine learning model processor, comprising:
• determining that the second executable binary requires a number of operations that, when added to the number of operations required by the first executable binary, does not exceed a number of operations available on the special purpose machine learning model processor;
• and
• allocating resources for the first and second machine executable binaries to execute together on the special purpose machine learning model processor.