Determining data representative of bias within a model

A computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computing device to cause the computing device to:

• obtain, as input, (i) a first dataset on which a model was trained, wherein the first dataset contains one or more protected attributes, and (ii) a second dataset on which the model was trained, wherein the one or more protected attributes have been removed from the second dataset;
• identify, for each of the one or more protected attributes in the first dataset, one or more attributes in the second dataset correlated therewith;
• determine bias among at least a portion of the identified correlated attributes, wherein said determining comprises:
• mapping at least two classes of data points associated with the correlated attributes in the second dataset to a set of values associated with the one or more protected attributes in the first dataset;
• and
• identifying one or more instances of bias by observing a change to one or more of the values in the mappings in response to modifying one or more class designations among the data points in the mappings;
• output, to at least one user, identifying information pertaining to the one or more instances of bias;
• and
• perform at least one validity analysis on the one or more identified instances of bias, wherein said performing the at least one validity analysis comprises:
• training, using at least one training dataset related to the one or more identified instances of bias, and implementing one or more of a nearest neighbors classifier and a nearest neighbors regressor, wherein implementing the one or more of a nearest neighbors classifier and a nearest neighbors regressor comprises identifying, using the one or more of a nearest neighbors classifier and a nearest neighbors regressor, one or more data points from the at least one training dataset which are within a predetermined proximity to at least a portion of the data points in the mappings with one or more modified class designations associated with the one or more instances of bias;
• and
• calculating at least one weighted distance between (i) the one or more data points from the at least one training dataset which are within a predetermined proximity to at least a portion of the data points in the mappings and (ii) the at least a portion of the data points in the mappings, wherein each weight represents a normalized correlation coefficient between at least one of the correlated attributes and at least one of the one or more protected attributes.

A system comprising:

• a memory;
• and
• at least one processor operably coupled to the memory and configured for:
• obtaining, as input, (i) a first dataset on which a model was trained, wherein the first dataset contains one or more protected attributes, and (ii) a second dataset on which the model was trained, wherein the one or more protected attributes have been removed from the second dataset;
• identifying, for each of the one or more protected attributes in the first dataset, one or more attributes in the second dataset correlated therewith;
• determining bias among at least a portion of the identified correlated attributes, wherein said determining comprises:
• mapping at least two classes of data points associated with the correlated attributes in the second dataset to a set of values associated with the one or more protected attributes in the first dataset;
• and
• identifying one or more instances of bias by observing a change to one or more of the values in the mappings in response to modifying one or more class designations among the data points in the mappings;
• outputting, to at least one user, identifying information pertaining to the one or more instances of bias;
• and
• performing at least one validity analysis on the one or more identified instances of bias, wherein said performing the at least one validity analysis comprises:
• training, using at least one training dataset related to the one or more identified instances of bias, and implementing one or more of a nearest neighbors classifier and a nearest neighbors regressor, wherein implementing the one or more of a nearest neighbors classifier and a nearest neighbors regressor comprises identifying, using the one or more of a nearest neighbors classifier and a nearest neighbors regressor, one or more data points from the at least one training dataset which are within a predetermined proximity to at least a portion of the data points in the mappings with one or more modified class designations associated with the one or more instances of bias;
• and
• calculating at least one weighted distance between (i) the one or more data points from the at least one training dataset which are within a predetermined proximity to at least a portion of the data points in the mappings and (ii) the at least a portion of the data points in the mappings, wherein each weight represents a normalized correlation coefficient between at least one of the correlated attributes and at least one of the one or more protected attributes.

A computer-implemented method comprising:

• obtaining (i) a first dataset on which a model was trained, wherein the first dataset contains one or more protected attributes, and (ii) a second dataset on which the model was trained, wherein the one or more protected attributes have been removed from the second dataset;
• identifying, for each of the one or more protected attributes, one or more attributes in the second dataset correlated therewith by cross-validating, in connection with at least a portion of the attributes in the second data set and the one or more protected attributes in the first dataset, a logistic regression model for one or more categorical features and a linear regression model for one or more numerical features;
• determining bias among at least a portion of the correlated attributes by:
• mapping two classes of data points associated with the correlated attributes in the second dataset to a set of values associated with the one or more protected attributes in the first dataset, wherein the two classes of data comprise a majority class and a minority class, distinguished in accordance with a predetermined threshold value;
• and
• identifying one or more instances of bias by observing a change to one or more of the values in the mappings in response to modifying one or more class designations among the data points in the mappings;
• carrying out one or more automated actions in connection with the one or more instances of bias;
• and
• performing at least one validity analysis on the one or more identified instances of bias, wherein said performing the at least one validity analysis comprises:
• training, using at least one training dataset related to the one or more identified instances of bias, and implementing one or more of a nearest neighbors classifier and a nearest neighbors regressor, wherein implementing the one or more of a nearest neighbors classifier and a nearest neighbors regressor comprises identifying, using the one or more of a nearest neighbors classifier and a nearest neighbors regressor, one or more data points from the at least one training dataset which are within a predetermined proximity to at least a portion of the data points in the mappings with one or more modified class designations associated with the one or more instances of bias;
• and
• calculating at least one weighted distance between (i) the one or more data points from the at least one training dataset which are within a predetermined proximity to at least a portion of the data points in the mappings and (ii) the at least a portion of the data points in the mappings, wherein each weight represents a normalized correlation coefficient between at least one of the correlated attributes and at least one of the one or more protected attributes;
• wherein the method is carried out by at least one computing device.