In-production model optimization

A method of optimizing a model deployed into production on an external system comprising:

• monitoring performance of the deployed model, wherein the monitoring includes receiving an output produced by the deployed model by processing an input;
• detecting at least one condition that necessitates generating a model evaluation trigger to evaluate a performance of at least the deployed model, wherein the at least one condition includes one of a date criterion or a data criterion;
• generating the model evaluation trigger upon detecting the at least one condition;
• calculating a model optimization function for each of a top K models, wherein K is a natural number and the top K models form a subset of K models selected from a plurality of models stored to a model repository, the selection being based on descending order of corresponding model optimization function values;
• determining that at least one model of the top K models has higher model optimization function value than the deployed model;
• and
• replacing the deployed model in the external system with the at least one model having the higher model optimization function value than the deployed model.

A non-transitory processor-readable storage medium comprising machine-readable instructions that cause a processor to:

• access output data of a ML model deployed in an external system wherein the ML model produces the output data based on input data received at the external system;
• generate a model evaluation trigger that initiates a performance evaluation of each of a plurality of ML models that include ML models stored on a model repository and the deployed ML model;
• calculate a model optimization function for each of the plurality of ML models, wherein the model optimization function is obtained as a weighted combination of different metrics;
• identify a ML model from the plurality of ML models with a highest value of the model optimization function for deployment to the external system;
• replace the ML model deployed to the external system with the ML model from the model repository having the highest value of the model optimization function if the ML model with the highest value of the model optimization function is different from the ML model deployed to the external system;
• and
• continue to use the ML model deployed to the external system for processing the input data to produce the output data if the ML model deployed to the external system has the highest value of the model optimization function.