A non-transitory computer readable storage medium storing executable computer program instructions for iteratively generating data modification operators to reduce bias within model input data, the computer program instructions when executed by one or more processors causing the one or more processors to perform steps comprising:
- receiving a set of metric values of one or more model output metrics in a set of outputs generated by a first artificial intelligence (AI) model,
- wherein receiving the set of metric values of the one or more model output metrics in the set of outputs comprises:
- for a first subset of the set of outputs that corresponds to a first subset of an initial set of corresponding inputs for the first AI model that shares a common characteristic, identifying a statistical difference between the first subset of the set of outputs and a second subset of the set of outputs that corresponds to a second subset of the initial set of corresponding inputs for the first AI model,
- wherein the second subset of the initial set of corresponding inputs does not share the common characteristic;
- and
- generating the set of metric values based on the statistical difference between the first subset of the set of outputs and the second subset of the set of outputs;
- and
- responsive to failure of the set of metric values in satisfying a data bias threshold, updating the initial set of corresponding inputs for the first AI model via iteratively performing operations comprising:
- generating a set of data modification operators by inputting, into a second AI model, the set of metric values of the one or more model output metrics to receive an output comprising one or more operations for adjusting the initial set of corresponding inputs for the first AI model by performing one or more of:
- (1) removing the common characteristic from the set of outputs, (2) generating a representation of a new characteristic of the set of outputs, or (3) applying a compensation value on at least one input of the initial set of corresponding inputs, wherein the second AI model is modified based on distortions detected in the set of outputs;
- generating a modified set of corresponding inputs for a third AI model based on the initial set of corresponding inputs for the first AI model, wherein generating the modified set of corresponding inputs for the third AI model includes:
- generating a duplicate set of corresponding inputs for the third AI model that includes copies of at least a portion of input data items in the initial set of corresponding inputs for the first AI model;
- and
- applying the one or more operations specified by the set of data modification operators to the copies of at least the portion of the input data items in the duplicate set of corresponding inputs for the third AI model to create modified copies;
- applying the third AI model to the modified set of corresponding inputs for the third AI model to generate a set of modified outputs, wherein the set of modified outputs comprises a lower measurement of bias compared to the set of outputs;
- and
- updating the set of metric values using one or more model output metrics in the set of modified outputs that are based on an updated statistical difference between subsets of modified outputs within the set of modified outputs,
- wherein the one or more model output metrics in the set of modified outputs are different than the one or more model output metrics in the set of outputs;
- and
- responsive to detecting that the updated set of metric values satisfies the data bias threshold, storing the set of modified outputs as generated outputs for the first AI model.