A non-transitory computer readable storage medium storing program code which, when executed by a processor, performs a computer-implemented algorithm comprising:
- training a plurality of predictor machine learning models (MLMs) using a first data set to predict a first plurality of classifications of a first plurality of data items in the first data set, to obtain a plurality of trained predictor MLMs;
- processing a second plurality of data items of a second data set by the plurality of trained predictor MLMs to predict a second plurality of classifications for the second plurality of data items, wherein the second plurality of classifications were predicted by the plurality of trained predictor MLMs and wherein the second data set includes having the second plurality of data items similar to the first plurality of data items in the first data set;
- training a plurality of confidence MLMs using the second plurality of classifications, to predict a first plurality of confidences that the second plurality of classifications is correct, to obtain a plurality of trained confidence MLMs;
- generating an aggregated ranked list of classes based on a third plurality of classifications outputted by the plurality of trained predictor MLMs and a second plurality of confidences that the third plurality of classifications is correct, outputted by the plurality of trained confidence MLMs;
- and
- training an ensemble confidence MLM using the aggregated ranked list of classes to generate a trained ensemble confidence MLM, wherein the trained ensemble confidence MLM is trained to predict a corresponding selected classification for each corresponding data item in a training data set containing the second plurality of data items similar to the first plurality of data items;
- re-training the plurality of trained predictor MLMs using a combination of the first data set, a second data set, and the training data set to form a plurality of re-trained predictor MLMs;
- deploying, to an enterprise environment, the trained ensemble confidence MLM and the plurality of re-trained predictor MLMs;
- receiving a new data set comprising a plurality of new data items;
- predicting, using the plurality of re-trained predictor MLMs, a plurality of new classifications for the plurality of new data items;
- predicting, using the plurality of trained confidence MLMs, a plurality of new confidences that the plurality of new classifications is correct;
- generating a new aggregated ranked list of classes using a combination of the plurality of new classifications and the plurality of new confidences;
- and
- predicting, using the trained ensemble confidence MLM operating on the new aggregated ranked list of classes, a plurality of selected new classifications for the plurality of new data items.