System and method for improving chatbot training dataset

A system for improving a training dataset, said training dataset comprising one or more utterances-intent pairs, the system comprising:

• a memory storing program instructions;
• at least one processor configured to execute program instructions stored in the memory;
• and an error detection engine executed by the at least one processor, and configured to:
• train a plurality of machine learning models with the training dataset to obtain a diverse set of trained Machine Learning (ML) models;
• feed each utterance of the one or more utterances-intent pairs as an input to the diverse set of trained ML models to obtain respective intent predictions for each utterance;
• evaluate a probability of error associated with each utterances-intent pair of the training dataset based on an analysis of the respective intent predictions for each utterance, wherein a mismatch during mapping the intent prediction for each utterance from each of the diverse set of ML models with the intent associated with the utterance in the training dataset and a similarity score (S) associated with the intent predictions for each utterance less than or equal to a predefined similarity-threshold (ST) is indicative of a high probability of error, the similarity score (S) is representative of percentage of ML models out of the diverse set of ML models providing similar intent predictions for same utterance, a mismatch during the mapping and the similarity score (S) greater than or equal to the predefined similarity-threshold (ST) is indicative of a high probability of error, a match during the mapping and the similarity score (S) less than or equal to the predefined similarity-threshold (ST) is indicative of a high probability of error, and a match during the mapping and the similarity score (S) greater than or equal to the predefined similarity-threshold (ST) is indicative of a low probability of error;
• and
• generate a set of improvement recommendations associated with each utterances-intent pair of the training dataset based on the evaluated probability of error.

A computer program product comprising:

• a non-transitory computer-readable medium having computer-readable program code stored thereon, the computer-readable program code comprising instructions that, when executed by a processor, cause the processor to:
• train a plurality of machine learning models with a training dataset to obtain a diverse set of trained Machine Learning (ML) models, wherein the training dataset comprises one or more utterances-intent pairs;
• feed each utterance of the one or more utterances-intent pairs as an input to the diverse set of trained ML models to obtain respective intent predictions for each utterance;
• evaluate a probability of error associated with each utterances-intent pair of the training dataset based on an analysis of the respective intent predictions for each utterance, wherein a mismatch during mapping of the intent prediction for each utterance from each of the diverse set of ML models with the intent associated with said utterance in the training dataset and a similarity score (S) associated with the intent predictions for each utterance less than or equal to a predefined similarity-threshold (ST) is indicative of a high probability of error, the similarity score (S) is representative of percentage of ML models out of the diverse set of ML models providing similar intent predictions for same utterance, a mismatch during the mapping and the similarity score (S) greater than or equal to the predefined similarity-threshold (ST) is indicative of a high probability of error, a match during the mapping and the similarity score (S) less than or equal to the predefined similarity-threshold (ST) is indicative of a high probability of error, and a match during the mapping and the similarity score (S) greater than or equal to the predefined similarity-threshold (ST) is indicative of a low probability of error;
• and
• generate a set of improvement recommendations associated with each utterances-intent pair of the training dataset based on at least the evaluated probability of error.