Automatic detection of labeling errors

A system comprising:

• one or more computer processors;
• and
• one or more computer-readable mediums storing instructions that, when executed by the one or more computer processors, cause the system to perform operations comprising:
• determining, using a classification model, a first classification label for a data item;
• determining that the first classification label determined for the data item by the classification model is different than a second classification label assigned to the data item by a human labeler;
• in response to determining that the first classification label determined for the data item by the classification model is different than the second classification label assigned to the data item by a human labeler, determining, using a model interpretability technique, a list of features that contributed to the classification model determining the first classification label for the data item;
• and
• determining, based on the list of features that contributed to the classification model, a probability value indicating a likelihood that the first classification label determined by the classification model properly classifies the data item.

A non-transitory computer-readable medium storing instructions that, when executed by one or more computer processors of a computing system, cause the computing system to perform operations comprising:

• determining, using a classification model, a first classification label for a data item;
• determining that the first classification label determined for the data item by the classification model is different than a second classification label assigned to the data item by a human labeler;
• in response to determining that the first classification label determined for the data item by the classification model is different than the second classification label assigned to the data item by a human labeler, determining, using a model interpretability technique, a list of features that contributed to the classification model determining the first classification label for the data item;
• and
• determining, based on the list of features that contributed to the classification model, a probability value indicating a likelihood that the first classification label determined by the classification model properly classifies the data item.