Methods and apparatus for machine learning engine optimization

A non-transitory computer readable storage medium comprising instructions which, when executed, cause one or more processors to at least:

• compute a first selection metric based on a first set of ordered output values from a first candidate machine learning engine and a set of reference data values;
• compute a second selection metric based on a second set of ordered output values from a second candidate machine learning engine and the set of reference data values;
• and
• determine whether to replace an active machine learning engine with at least one of the first candidate machine learning engine or the second candidate machine learning engine based on the first selection metric and the second selection metric.

A method comprising:

• computing, by executing an instruction with a processor, a first selection metric based on a first set of ordered output values from a first candidate machine learning engine and a set of reference data values;
• computing, by executing an instruction with the processor, a second selection metric based on a second set of ordered output values from a second candidate machine learning engine and the set of reference data values;
• and
• determining, by executing an instruction with the processor, whether to replace an active machine learning engine with at least one of the first candidate machine learning engine or the second candidate machine learning engine based on the first selection metric and the second selection metric.