System and method for alerts for missing coverage of chatbot conversation messages

A system comprising:

• a processor;
• a data bus coupled to the processor;
• and
• a computer-usable medium embodying computer program code, the computer-usable medium being coupled to the data bus, the computer program code used for identifying areas to improve an interactive conversational system executable by the processor and configured for:
• receiving a stream of conversational interactions C (C1, C2, . . . , Cn) between users and the interactive conversational system;
• monitoring instances of the conversational interactions C (C1, C2, . . . , Cn) for intents that do not exist for a set of related messages or questions;
• applying periodically an intent clustering model to the stream to form an incremental clustering based on a set of derived intents to form a mapping from a first set of conversational characteristics to a first set of intents and a second set of conversational characteristics to a first set of unclear intents that include the intents that do not exist, wherein changes are made to a set of candidate intents relevant to a set of incoming utterances, compute semantic similarity between the utterances;
• computing semantic similarity between the utterances and centroids of exiting clusters;
• adding an utterance to a first cluster that matches a condition;
• creating a new cluster if there is no cluster matching the condition, wherein the utterance is the centroid of the new cluster, wherein an intent clustering algorithm is applied to the set of incoming utterances and trained based on previous selections of subject matter experts;
• and
• providing information related to the second set of conversation characteristics.

A non-transitory, computer-readable storage medium embodying computer program code, the computer program code comprising computer executable instructions configured for:

• receiving a stream of conversational interactions C (C1, C2, . . . , Cn) between users and the interactive conversational system;
• monitoring instances of the conversational interactions C (C1, C2, . . . , Cn) for intents that do not exist for a set of related messages or questions;
• applying periodically an intent clustering model to the stream to form an incremental clustering based on a set of derived intents to form a mapping from a first set of conversational characteristics to a first set of intents and a second set of conversational characteristics to a first set of unclear intents that include the intents that do not exist, wherein changes are made to a set of candidate intents relevant to a set of incoming utterances;
• computing semantic similarity between the utterances and centroids of exiting clusters;
• adding an utterance to a first cluster that matches a condition;
• creating a new cluster if there is no cluster matching the condition, wherein the utterance is the centroid of the new cluster, wherein an intent clustering algorithm is applied to the set of incoming utterances and trained based on previous selections of subject matter experts;
• and
• providing information related to the second set of conversation characteristics.