System and method for automated testing of customer support chatbots

A system for evaluating performance of a chatbot, the system comprising:

• a processor;
• and
• a memory, wherein the memory includes instructions that, when executed by the processor, cause the processor to perform:
• identifying a test scenario comprising a request and an expected outcome;
• initiating an automated conversation between a first machine learning model and the chatbot based on the test scenario;
• storing a recording of the automated conversation;
• providing the recording of the automated conversation and the test scenario to a second machine learning model;
• and
• receiving an evaluation of the automated conversation from the second machine learning model based on the recording of the automated conversation and the expected outcome.