Human-in-the-loop conflict resolution in a collaborative data labeling platform

A computer system for data labeling, comprising:

• one or more processors, one or more computer-readable memories, one or more computer-readable tangible storage medium, and program instructions stored on at least one of the one or more tangible storage medium for execution by at least one of the one or more processors via at least one of the one or more memories, wherein the computer system is capable of performing a method comprising:
• receiving a plurality of labeled data points;
• identifying one or more of the plurality of labeled data points with conflicting labels;
• determining that at least one of the one or more identified labeled data points exceeds one or more conflict thresholds;
• presenting the at least one of the one or more identified labeled data points exceeding the one or more conflict thresholds to a user;
• and
• receiving a conflict resolution from the user for the one or more identified labeled data points exceeding the one or more conflict thresholds, wherein the conflict resolution received from the user is utilized in training a machine learning model.

A computer program product for data labeling, comprising:

• one or more non-transitory computer-readable storage media and program instructions stored on at least one of the one or more tangible storage media, the program instructions executable by a processor to cause the processor to perform a method comprising:
• receiving a plurality of labeled data points;
• identifying one or more of the plurality of labeled data points with conflicting labels;
• determining that at least one of the one or more identified labeled data points exceeds one or more conflict thresholds;
• presenting the at least one of the one or more identified labeled data points exceeding the one or more conflict thresholds to a user;
• and
• receiving a conflict resolution from the user for the one or more identified labeled data points exceeding the one or more conflict thresholds, wherein the conflict resolution received from the user is utilized in training a machine learning model.