Identifying and Correcting Label Bias in Machine Learning

A computer-implemented method to reduce bias in a machine-learned classification model, the method comprising:

• obtaining, by one or more computing devices, a training dataset comprising a plurality of training examples, each training example comprising an example input and a respective example label applied to the example input, wherein the example labels of the training dataset exhibit a bias against one or more subgroups of the example inputs;
• initializing, by the one or more computing devices, a plurality of weights that are respectively associated with the plurality of training examples;
• for each of one or more training iterations:
• determining, by the one or more computing devices, one or more constraint violation values for the machine-learned classification model on the training dataset relative to one or more fairness constraints applied to the one or more subgroups of the example inputs;
• updating, by the one or more computing devices, one or more re-weighting control values respectively associated with the one or more fairness constraints based at least in part on the one or more constraint violation values;
• modifying, by the one or more computing devices, at least one of the plurality of weights associated with the plurality of training examples based at least in part on the one or more re-weighting control values to form a plurality of modified weights;
• and
• re-training, by the one or more computing devices, the machine-learned classification model using the training dataset weighted according to the plurality of modified weights.