System and method for managing deployment of related inference models
AssigneeDell Products L.P.
InventorsOFIR EZRIELEV, JEHUDA SHEMER, TOMER KUSHNIR
Life of this patent
Estimating the term…
Abstract
Methods and systems for managing inference models hosted by data processing systems are disclosed. To manage the inference models, inference models that may share computations with other inference models may be preferentially selected for deployment. Additionally, the inference models may be divided into portions to distribute the computation load across more data processing systems. The location of deployment of the portions of the inference models may be preferentially selected based on the locations of data sources and inference consumers.
Claims20
A method of managing inference models hosted by data processing systems, the method comprising:
- obtaining first location data for sources of data usable to obtain inferences;
- obtaining second location data for consumers of the inferences;
- obtain a first inference model based on a first goal of the consumers for deployment to the data processing systems;
- obtain a second inference model based on a second goal of the consumers for deployment to the data processing systems, the second inference model sharing at least one hidden layer with the first inference model;
- obtaining a deployment plan for the first inference model and the second inference model, the deployment plan specifying:
- a first deployment location for a shared portion of the first inference model and the second inference model based on the first location data,
- second deployment locations for independent portions of the first inference model and the second inference model based on the second location data, and
- that the shared portion is to distribute a partial processing result to both of the independent portions;
- deploying the first inference model and the second inference model to the data processing systems based on the deployment plan;
- and
- providing inferences to the consumers using the deployed first inference model, the deployed second inference model, and the sources of data.
A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations for managing inference models hosted by data processing systems, the operations comprising:
- obtaining first location data for sources of data usable to obtain inferences;
- obtaining second location data for consumers of the inferences;
- obtain a first inference model based on a first goal of the consumers for deployment to the data processing systems;
- obtain a second inference model based on a second goal of the consumers for deployment to the data processing systems, the second inference model sharing at least one hidden layer with the first inference model;
- obtaining a deployment plan for the first inference model and the second inference model, the deployment plan specifying:
- a first deployment location for a shared portion of the first inference model and the second inference model based on the first location data,
- second deployment locations for independent portions of the first inference model and the second inference model based on the second location data, and
- that the shared portion is to distribute a partial processing result to both of the independent portions;
- deploying the first inference model and the second inference model to the data processing systems based on the deployment plan;
- and
- providing inferences to the consumers using the deployed first inference model, the deployed second inference model, and the sources of data.
A data processing system, comprising:
- a processor;
- and
- a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations for managing a distribution of inference models hosted by data processing systems, the operations comprising:
- obtaining first location data for sources of data usable to obtain inferences;
- obtaining second location data for consumers of the inferences;
- obtain a first inference model based on a first goal of the consumers for deployment to the data processing systems;
- obtain a second inference model based on a second goal of the consumers for deployment to the data processing systems, the second inference model sharing at least one hidden layer with the first inference model;
- obtaining a deployment plan for the first inference model and the second inference model, the deployment plan specifying:
- a first deployment location for a shared portion of the first inference model and the second inference model based on the first location data,
- second deployment locations for independent portions of the first inference model and the second inference model based on the second location data, and
- that the shared portion is to distribute a partial processing result to both of the independent portions;
- deploying the first inference model and the second inference model to the data processing systems based on the deployment plan;
- and
- providing inferences to the consumers using the deployed first inference model, the deployed second inference model, and the sources of data.
Dependencies are inferred from the claim text.