Resource prediction system for executing machine learning models

A system comprising:

• non-transitory memory storing instructions executable to select a suggested machine learning model executing on a hardware platform under constraints;
• and
• a processor configured to execute the instructions to:
• obtain input data including a targeted objective and the constraints;
• determine configurations of candidate hardware platforms based on the input data;
• virtually execute, in a first virtualization process, the candidate machine learning models defined by a hyper-parameter set corresponding to the configurations based on the input data for a selection of machine learning model;
• obtain a virtual performance metrics set of each respective candidate machine learning model from virtual execution of the candidate machine learning models;
• generate an evaluation score of each respective candidate machine learning model based on a similarity of the virtual performance metrics set of each respective candidate machine learning model to the targeted objective;
• select a deployable machine learning model having the evaluation score that meets a predetermined criterion from among the candidate machine learning models;
• virtually execute, in a second virtualization process, each of candidate machine learning models on the candidate hardware platforms according to the constraints, the second virtualization process being performed by a hardware performance model to simulate the deployable machine learning model, the hardware performance model being trained based on probabilistic programming and comprising at least one generative model;
• generate an assessment report of the virtual performance metrics set of the deployable machine learning model executed on each of the candidate hardware platforms;
• select the suggested machine learning model meeting a predetermined criterion from among the candidate machine learning models, the suggested machine learning model probabilistically satisfying the targeted objective under the constraints when combined with the candidate hardware platforms for execution;
• output a model of an implementation of resource prediction based on the suggested machine learning model;
• and
• automatically perform, using a resource prediction twin, the resource prediction and allocation of tasks associated with the resource prediction according to the outputted model, wherein the resource prediction twin includes a generative model that is trained, using probabilistic programming, on parameters of an inference model that is initialized with virtual parameters to generate real parameters as input to the outputted model;
• and
• based on the real parameter as input to the outputted model, dynamically assigning a predetermined portion of required tasks to a first hardware device from the candidate hardware platforms and other portion of required tasks to a second hardware device depending on the performance of each respective hardware device in order to optimally accomplish the targeted objective.

A computer-implemented method performed by a processor configured to execute instructions to select a suggested hardware platform executing a machine learning model under constraints stored in non-transitory memory, the method comprising:

• obtaining input data including a targeted objective and the constraints;
• determining configurations of candidate machine learning models based on the input data;
• virtually executing, in a first virtualization process, the candidate machine learning models defined by a hyper-parameter set corresponding to the configurations based on the input data for a selection of machine learning model;
• obtaining a virtual performance metrics set of each respective candidate machine learning model from virtual execution of the candidate machine learning models;
• generating an evaluation score of each respective candidate machine learning model based on a similarity of the virtual performance metrics set of each respective candidate machine learning model to the targeted objective;
• selecting a deployable machine learning model having the evaluation score that meets a predetermined criterion from among the candidate machine learning models;
• virtually execute, in a second virtualization process, the deployable machine learning model selected from the candidate machine learning models on each of candidate hardware platforms according to the constraints, the second virtualization process being performed by a hardware performance model to simulate the deployable machine learning model, the hardware performance model being trained based on probabilistic programming and comprising at least one generative model;
• generating an assessment report of the virtual performance metrics set of the deployable machine learning model executed on each of the candidate hardware platforms;
• selecting the suggested hardware platform meeting the predetermined criterion from among the candidate hardware platforms for running the selected machine learning model, the suggested hardware platform probabilistically satisfying the targeted objective under the constraints when combined with the deployable machine learning model for execution;
• outputting a model of an implementation of resource prediction based on the suggested hardware platform;
• automatically performing, using a resource prediction twin, the resource prediction and allocation of tasks associated with the resource prediction according to the outputted model, wherein the resource prediction twin includes a generative model that is trained, using probabilistic programming, on parameters of an inference model that is initialized with virtual parameters to generate real parameters as input to the outputted model;
• and
• based on the real parameter as input to the outputted model, dynamically assigning a predetermined portion of required tasks to a first hardware device from the candidate hardware platforms and other portion of required tasks to a second hardware device depending on the performance of each respective hardware device in order to optimally accomplish the targeted objective.

A computer-implemented method performed by a processor configured to execute instructions to select a suggested machine learning model executing on a hardware platform under constraints stored in non-transitory memory, the method comprising:

• obtaining by the processor, input data including a targeted objective and the constraints;
• determining by the processor, configurations of candidate hardware platforms based on the input data;
• virtually executing, in a first virtualization process, the candidate machine learning models defined by a hyper-parameter set corresponding to the configurations based on the input data for a selection of machine learning model;
• obtaining a virtual performance metrics set of each respective candidate machine learning model from virtual execution of the candidate machine learning models;
• generating an evaluation score of each respective candidate machine learning model based on a similarity of the virtual performance metrics set of each respective candidate machine learning model to the targeted objective;
• selecting a deployable machine learning model having the evaluation score that meets a predetermined criterion from among the candidate machine learning models;
• virtually executing by the processor, in a second virtualization process, each of candidate machine learning models on the candidate hardware platforms according to the constraints, the second virtualization process being performed by a hardware performance model to simulate the deployable machine learning model, the hardware performance model being trained based on probabilistic programming and comprising at least one generative model;
• generating an assessment report of the virtual performance metrics set of the deployable machine learning model executed on each of the candidate hardware platforms;
• selecting by the processor, the suggested machine learning model meeting a predetermined criterion from among the candidate machine learning models, the suggested machine learning model probabilistically satisfying the targeted objective under the constraints when combined with the candidate hardware platforms for execution;
• outputting, by the processor, a model of an implementation of resource prediction based on the candidate hardware platforms and the suggested machine learning model;
• and
• automatically performing, using a resource prediction twin, the resource prediction and allocation of tasks associated with the resource prediction according to the outputted model, wherein the resource prediction twin includes a generative model that is trained, using probabilistic programming, on parameters of an inference model that is initialized with virtual parameters to generate real parameters as input to the outputted model, and
• based on the real parameter as input to the outputted model, dynamically assigning a predetermined portion of required tasks to a first hardware device from the candidate hardware platforms and other portion of required tasks to a second hardware device depending on the performance of each respective hardware device in order to optimally accomplish the targeted objective.