Trustworthiness of artificial intelligence models in presence of anomalous data

A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computing device to cause the computing device to:

• obtain (i) a trained machine learning model and (ii) a set of data points used to train said machine learning model;
• determine one or more anomalous data points in said set of data points;
• for a given one of said anomalous data points, identify at least one attribute of said given anomalous data point that decreases confidence with respect to at least one output of said machine learning model;
• determine that a root cause of said decreased confidence corresponds to one of:
• a class imbalance issue related to said at least one attribute, a confused class issue related to said at least one attribute, a low density issue related to said at least one attribute, and an adversarial issue related to said at least one attribute;
• and
• perform one or more steps to improve said confidence of said machine learning model based at least in part on said determined root cause.

A system comprising:

• a memory;
• and
• at least one processor operably coupled to the memory and configured for:
• obtaining (i) a trained machine learning model and (ii) a set of data points used to train said machine learning model;
• determining one or more anomalous data points in said set of data points;
• for a given one of said anomalous data points, identifying at least one attribute of said given anomalous data point that decreases confidence with respect to at least one output of said machine learning model;
• determining that a root cause of said decreased confidence corresponds to one of:
• a class imbalance issue related to said at least one attribute, a confused class issue related to said at least one attribute, a low density issue related to said at least one attribute, and an adversarial issue related to said at least one attribute;
• and
• performing one or more steps to improve said confidence of said machine learning model based at least in part on said determined root cause.

A computer-implemented method, the method comprising:

• obtaining (i) a machine learning model, (ii) a set of training data used to train said machine learning model, and (iii) at least one data point in said set of training data identified as an anomaly;
• determining a root cause of behavior of said machine learning model that caused said at least one data point to be an anomaly, wherein determining said root cause comprises one or more of:
• (i) determining a class imbalance in said set of training data;
• (ii) identifying that said at least one data point is an adversarial data point;
• (iii) identifying that said at least one data point is a confused class data point;
• and (iv) identifying that said at least one data point corresponds to a low density region in said set of training data;
• adjusting said set of training data, based on said root cause, by at least one of:
• (i) relabeling at least a portion of said training data and (ii) adding additional training data to said set of training data;
• and
• re-training said machine learning model based on the adjusted set of training data to remove the behavior caused by said at least one anomaly;
• wherein the method is carried out by at least one computing device.