System, Method, and Computer Program Product for Identifying Weak Points in a Predictive Model

A system, comprising:

• at least one processor programmed and/or configured to:
• obtain a plurality of features associated with a plurality of samples and a plurality of labels for the plurality of samples;
• generate a plurality of first predictions for the plurality of samples by providing, as input to a first machine learning model, a first subset of features of the plurality of features, and receiving, as output from the first machine learning model, the plurality of first predictions for the plurality of samples;
• generate a plurality of second predictions for the plurality of samples by providing, as input to a second machine learning model, a second subset of features of the plurality of features, and receiving, as output from the second machine learning model, the plurality of second predictions for the plurality of samples;
• generate, based on the plurality of first predictions, the plurality of second predictions, the plurality of labels, and a plurality of groups of samples of the plurality of samples;
• determine, based on the plurality of groups of samples, a first success rate associated with the first machine learning model and a second success rate associated with the second machine learning model;
• and
• identify, based on the first success rate and the second success rate, a weak point in the second machine learning model associated with a first portion of samples of the plurality of samples including a same first value for a same first feature of the plurality of features and for which the first success rate associated with the first machine learning model is different than the second success rate associated with the second machine learning model.

A computer program product comprising at least one non-transitory computer-readable medium including program instructions that, when executed by at least one processor, cause the at least one processor to:

• obtain a plurality of features associated with a plurality of samples and a plurality of labels for the plurality of samples;
• generate a plurality of first predictions for the plurality of samples by providing, as input to a first machine learning model, a first subset of features of the plurality of features, and receiving, as output from the first machine learning model, the plurality of first predictions for the plurality of samples;
• generate a plurality of second predictions for the plurality of samples by providing, as input to a second machine learning model, a second subset of features of the plurality of features, and receiving, as output from the second machine learning model, the plurality of second predictions for the plurality of samples;
• generate, based on the plurality of first predictions, the plurality of second predictions, the plurality of labels, and a plurality of groups of samples of the plurality of samples;
• determine, based on the plurality of groups of samples, a first success rate associated with the first machine learning model and a second success rate associated with the second machine learning model;
• and
• identify, based on the first success rate and the second success rate, a weak point in the second machine learning model associated with a first portion of samples of the plurality of samples including a same first value for a same first feature of the plurality of features and for which the first success rate associated with the first machine learning model is different than the second success rate associated with the second machine learning model.