Data-driven taxonomy for annotation resolution

A system comprising:

• one or more processors;
• and
• memory storing thereon instructions that, as a result of being executed by the one or more processors, cause the system to perform operations comprising:
• receiving ongoing conversation data that includes exchanged messages;
• applying a machine-learning model to the ongoing conversation data to generate a set of annotations for the messages, wherein an annotation of the set of annotations corresponds to a particular message, and wherein the annotation of the particular message includes a set of confidence scores corresponding to classifications;
• generating an estimated amount of confusion present in the classifications by analyzing the annotation and the set of confidence scores, wherein the estimated amount of confusion represents a level of confusion between a pair of the classifications;
• dynamically generating in real-time a graphical representation that includes a set of nodes and a set of edges, wherein two nodes represent the pair of classifications, wherein an edge connects the two nodes and represents the estimated amount of confusion between the pair of classifications, and wherein the graphical representation is generated as the ongoing conversation data continues to be received;
• receiving input corresponding to an interaction with the graphical representation;
• modifying a classification of the pair of the classifications according to the interaction, wherein modifying includes decreasing the estimated amount of confusion between the pair of classifications;
• updating the machine-learning model based on the modified classification;
• and
• applying the updated machine-learning model to the ongoing conversation data to generate an updated set of annotations for the messages, wherein an updated annotation includes an updated confidence score corresponding to the modified classification.

A non-transitory, computer-readable storage medium storing thereon executable instructions that, as a result of being executed by one or more processors of a computer system, cause the computer system to perform operations comprising:

• receiving ongoing conversation data that includes exchanged messages;
• applying a machine-learning model to the ongoing conversation data to generate a set of annotations for the messages, wherein an annotation of the set of annotations corresponds to a particular message, and wherein the annotation of the particular message includes a set of confidence scores corresponding to classifications;
• generating an estimated amount of confusion present in the classifications by analyzing the annotation and the set of confidence scores, wherein the estimated amount of confusion represents a level of confusion between a pair of the classifications;
• dynamically generating in real-time a graphical representation that includes a set of nodes and a set of edges, wherein two nodes represent the pair of classifications, wherein an edge connects the two nodes and represents the estimated amount of confusion between the pair of classifications, and wherein the graphical representation is generated as the ongoing conversation data continues to be received;
• receiving input corresponding to an interaction with the graphical representation;
• modifying a classification of the pair of the classifications according to the interaction, wherein modifying includes decreasing the estimated amount of confusion between the pair of classifications;
• updating the machine-learning model based on the modified classification;
• and
• applying the updated machine-learning model to the ongoing conversation data to generate an updated set of annotations for the messages, wherein an updated annotation includes an updated confidence score corresponding to the modified classification.