Assessing the structural quality of conversations

A computer system comprising:

• one or more computer processors, one or more readable storage media, and program instructions stored on the one or more computer readable storage media for execution by at least one of the one or more computer processors, the program instructions comprising:
• program instructions to receive an input of a conversation, in real time, between at least a first user and an automated response system, wherein the automated response system comprises an application that performs an automated task;
• program instructions to analyze an utterance of the conversation to identify a dialog act attribute, an emotion attribute, and a tone attribute;
• program instructions to annotate the utterance of the conversation with the dialog act attribute, the emotion attribute, and the tone attribute;
• program instructions to validate the conversation based on the annotated attributes in comparison with a threshold, wherein program instructions to validate the conversation comprise:
• program instructions to compare each annotated utterance in the conversation to a known database of best practice annotated conversations;
• program instructions to determine a number of violations and a type of each violation using an algorithm selected from the group consisting of:
• a sequence alignment algorithm, an information retrieval method, a sequence-to-one model, and a weighted edit distance;
• program instructions to score each annotated utterance based on the number of violations, based on the comparison of each annotated utterance to the known database of best practice annotated conversations;
• program instructions to rank each utterance based on the scoring;
• and
• program instructions to suggest a response for a second user to make to the first user based on each ranked utterance;
• program instructions to store the annotated conversation and the validation of the conversation;
• program instructions to determine that the validation of the conversation indicates that the conversation has not met the threshold;
• program instructions to transfer the first user to a second user;
• program instructions to retrieve a plurality of stored annotated conversations;
• program instructions to analyze the plurality of stored annotated conversations, wherein the analysis determines a number of violations found in each annotated conversation;
• program instructions to rank each annotated conversation in the plurality of stored annotated conversations based on the analysis;
• and
• program instructions to send at least one recommendation to a third user for improving at least one annotated conversation in the plurality of stored annotated conversations, wherein the at least one recommendation includes a change to a pre-defined dialog.