System and method for improving machine learning models by detecting and removing inaccurate training data

A system, comprising:

• a processor;
• and
• a memory device that stores program code structured to cause the processor to:
• provide, as input to a machine learning (ML) model trained on a first portion of a plurality of data items and user-generated labels associated with the data items, a second portion of the data items;
• receive from the ML model ML model-generated labels for the data items in the second portion;
• compare the user-generated labels and the ML model-generated labels associated with same data items in the second portion;
• determine a confusion metric for each distinct pair of labels in the ML model-generated labels based on the compare to determine a plurality of confusion metrics;
• and
• identify an ML model-generated label of the ML model-generated labels as a problematic label based on the determined confusion metrics.

A computer-readable storage device having instructions recorded thereon that, when executed by a processor, implements a method comprising:

• providing, as input to a machine learning (ML) model trained on a first portion of a plurality of data items and user-generated labels associated with the data items, a second portion of the data items;
• receiving from the ML model ML model-generated labels for the data items in the second portion;
• comparing the user-generated labels and the ML model-generated labels associated with same data items in the second portion;
• determining a confusion metric for each distinct pair of labels in the ML model-generated labels based said comparing to determine a plurality of confusion metrics;
• and
• identifying an ML model-generated label of the ML model-generated labels as a problematic label based on the determined confusion metrics.