Multiple model injection for a deployment cluster
AssigneeHewlett Packard Enterprise Development LP
InventorKartik Mathur
Life of this patent
Estimating the term…
Abstract
Systems and methods are provided for servicing inference request by one of multiple machine learning models attached to a deployment cluster. The API server of a deployment cluster is not tightly coupled to any of multiple machine learning models attached to the deployment cluster. Upon receiving an inference request, the deployment cluster can retrieve the configuration parameters, including serialization formatting, for a target model identified in the inference request. The deployment cluster can utilize the retrieved parameters to service the inference request and return the results to a business system application.
Claims20
A management system comprising:
- one or more physical or virtualized computing devices communicatively coupled to a business system application and a plurality of deployment clusters;
- a processor;
- and
- a non-transitory computer-readable medium configured to execute instructions by the processor to cause the management system to:
- determine model-specific configuration parameters for a machine learning model;
- create a model-specific entry in a data structure associated with the machine learning model;
- identify a deployment cluster of the plurality of deployment clusters to attach the machine learning model;
- and
- transmit the model-specific configuration parameters to the identified deployment cluster.
A system comprising:
- a business system application encoded as instructions on a non-transitory computer-readable medium and executed on a processor;
- a plurality of deployment clusters comprising one or more physical or virtualized computing devices;
- and
- a management system comprising one or more physical or virtualized computing devices, communicatively coupled to the business system application and the plurality of deployment clusters, wherein the management system is configured to:
- determine model-specific configuration parameters for a machine learning model;
- create a model-specific entry in a data structure associated with the machine learning model;
- identify a deployment cluster of the plurality of deployment clusters to attach the machine learning model;
- and
- transmit the model-specific configuration parameters to the identified deployment cluster.
A non-transitory machine-readable storage medium storing instructions thereon that, when executed by a processor, cause the processor to:
- determine model-specific configuration parameters for a machine learning model;
- create a model-specific entry in a data structure associated with the machine learning model;
- identify a deployment cluster of a plurality of deployment clusters to attach the machine learning model;
- and
- transmit the model-specific configuration parameters to the identified deployment cluster.
Dependencies are inferred from the claim text.