System and method for managing inference model distributions in dynamic system

A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations for managing a distribution of inference models hosted by data processing systems, the operations comprising:

• obtaining condition data for the data processing systems, the condition data indicating how the data processing systems are likely to operate during a future period of time;
• obtaining a deployment plan for the inference models based on the condition data, the distribution, and inference model types available for deployment, the deployment plan specify an operating mode and a new distribution for the data processing system;
• prior to the future period of time, updating the inference models based on the deployment plan to obtain an updated distribution of the inference models;
• and
• obtaining, during the period of time, an inference using the updated distribution of the inference models and the operating mode, the operating mode indicating at least one automatically implemented change to the updated distribution of the inference models responsive to an occurrence of a condition.

A data processing system, comprising:

• a processor;
• and
• a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations for managing a distribution of inference models hosted by data processing systems, the operations comprising:
• obtaining condition data for the data processing systems, the condition data indicating how the data processing systems are likely to operate during a future period of time;
• obtaining a deployment plan for the inference models based on the condition data, the distribution, and inference model types available for deployment, the deployment plan specify an operating mode and a new distribution for the data processing system;
• prior to the future period of time, updating the inference models based on the deployment plan to obtain an updated distribution of the inference models;
• and
• obtaining, during the period of time, an inference using the updated distribution of the inference models and the operating mode, the operating mode indicating at least one automatically implemented change to the updated distribution of the inference models responsive to an occurrence of a condition.