Anomaly detection method for model outputs

A method comprising:

• receiving a user prompt indicating a request for data over a time period;
• generating a first plurality of tokens by:
• inputting, into a generative model, the user prompt to cause the generative model to generate an output based on the user prompt, the output comprising a plurality of text-based analytics and a plurality of visual analytics based on the user prompt;
• applying a text filter to the output to generate a modified output comprising the plurality of text-based analytics and excluding the plurality of visual analytics;
• and
• generating the first plurality of tokens based on the plurality of text-based analytics;
• generating a second plurality of tokens by:
• retrieving, based on the user prompt, a plurality of sources relating to the data requested by the user prompt;
• generating a plurality of queries to request, from the plurality of sources, the data over the time period;
• and
• generating the second plurality of tokens based on the data over the time period;
• performing a comparison of the first plurality of tokens and the second plurality of tokens;
• based on the comparison, detecting an anomaly in the output of the generative model;
• based on the detected anomaly, rejecting the output of the generative model;
• and
• automatically providing feedback to the generative model indicating one or more specific tokens of the first plurality of tokens that contributed to a match accuracy failing to satisfy an anomaly detection threshold.

A system comprising:

• a storage device;
• and
• one or more processors communicatively coupled to the storage device storing instructions thereon, that cause the one or more processors to:
• receive a request for data over a time period;
• generate a first plurality of tokens by causing the system to:
• input, into an artificial intelligence (AI) model, the user prompt to cause the AI model to generate an output based on the user prompt;
• and
• generate the first plurality of tokens based on the output;
• generate a second plurality of tokens by causing the system to:
• retrieve, based on the user prompt, a plurality of sources relating to the data requested by the user prompt;
• generate a plurality of queries to request, from the plurality of sources, the data over the time period;
• and
• generate the second plurality of tokens based on the data over the time period;
• perform a comparison of the first plurality of tokens and the second plurality of tokens;
• based on the comparison, flag the output of the AI model as an anomaly;
• and
• provide feedback to the generative model indicating one or more reasons for flagging the output of the AI model as the anomaly.