Multi-domain bias and hallucination evaluation systems and methods for large language models

A system comprising:

• a processing unit;
• and
• a storage device comprising instructions, which when executed by the processing unit, configure the processing unit to perform operations comprising:
• generating a test set of prompts;
• executing a generative artificial intelligence (GenAI) machine learning model using the test set of prompts;
• in response to the executing, receiving a plurality of generated answers;
• classifying the plurality of generated answers into a first group and a second group;
• calculating a percentage of generated answers in the first group compared to a total number of answers of the first group and second group;
• determining the percentage exceeds a value;
• based on the determining, updating a bias metric the GenAI machine learning model;
• and
• presenting the bias metric on a user interface.

A non-transitory computer-readable medium comprising instructions, which when executed by a processing unit, configure the processing unit to perform operations comprising:

• generating a test set of prompts;
• executing a generative artificial intelligence (GenAI) machine learning model using the test set of prompts;
• in response to the executing, receiving a plurality of generated answers;
• classifying the plurality of generated answers into a first group and a second group;
• calculating a percentage of generated answers in the first group compared to a total number of answers of the first group and second group;
• determining the percentage exceeds a value;
• based on the determining, updating a bias metric the GenAI machine learning model;
• and
• presenting the bias metric on a user interface.