Detecting and reducing bias in machine learning models

A computer program product comprising a computer readable storage medium having program code embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, and wherein the program code is readable and executable by a processor to perform a method comprising:

• inputting a plurality of input training data into a machine learning model in a machine learning system to generate an output of labeled output data;
• evaluating the labeled output data according to a consistency metric to associate the labeled output data with a corresponding consistency assessment;
• selecting each labeled output data having a consistency assessment indicating a consistency assessment that is greater than a predetermined threshold to form a mislabeled output data subset;
• identifying the mislabeled output data subset as being generated by the machine learning system based on a particular input training data from the plurality of input training data, wherein the particular input training data is labeled by a particular labeling person, and wherein the particular labeling person has a personal profile that identifies the particular labeling person as having personal knowledge of an appearance of an object that the machine learning system is being trained to recognize;
• redefining the mislabeled output data subset as an outlier labeled output data subset that correctly describes the object, wherein the mislabeled output data is redefined as the outlier labeled output data subset based on the personal profile of the labeling person;
• adjusting the machine learning system based on the outlier labeled output data subset, wherein the adjusted machine learning system is a trained machine learning system for identifying the object;
• determining that the trained machine learning system outputs multiple different categories that describe the object;
• comparing features from the personal profile of the particular labeling person to the multiple different categories that describe the object;
• ranking the multiple different categories that describe the object based on a relevance of the features from the personal profile to each of the multiple categories that describe the object;
• and
• presenting the multiple different categories in a ranked order based on the relevance of the features from the personal profile to each of the multiple categories that describe the object.

A computer system comprising one or more processors, one or more computer readable memories, and one or more computer readable non-transitory storage mediums, and program instructions stored on at least one of the one or more computer readable non-transitory storage mediums for execution by at least one of the one or more processors via at least one of the one or more computer readable memories, the stored program instructions executed to perform a method comprising:

• inputting a plurality of input training data into a machine learning model in a machine learning system to generate an output of labeled output data;
• evaluating the labeled output data according to a consistency metric to associate the labeled output data with a corresponding consistency assessment;
• selecting each labeled output data having a consistency assessment indicating a consistency assessment that is greater than a predetermined threshold to form a mislabeled output data subset;
• identifying the mislabeled output data subset as being generated by the machine learning system based on a particular input training data from the plurality of input training data, wherein the particular input training data is labeled by a particular labeling person, and wherein the particular labeling person has a personal profile that identifies the particular labeling person as having personal knowledge of an appearance of an object that the machine learning system is being trained to recognize;
• redefining the mislabeled output data subset as an outlier labeled output data subset that correctly describes the object, wherein the mislabeled output data is redefined as the outlier labeled output data subset based on the personal profile of the labeling person;
• and
• adjusting the machine learning system based on the outlier labeled output data subset, wherein the adjusted machine learning system is a trained machine learning system for identifying the object;
• determining that the trained machine learning system outputs multiple different categories that describe the object;
• comparing features from the personal profile of the particular labeling person to the multiple different categories that describe the object;
• ranking the multiple different categories that describe the object based on a relevance of the features from the personal profile to each of the multiple categories that describe the object;
• and
• presenting the multiple different categories in a ranked order based on the relevance of the features from the personal profile to each of the multiple categories that describe the object.