System for detecting errors in generative AI outputs

A computer-implemented method for detecting errors, hallucinations, and biases in outputs generated by generative artificial intelligence models, the method comprising:

• parsing, by a processor, a received Generative Artificial Intelligence (GenAI) output to generate a metadata string comprising one or more tags;
• applying, by the processor, a recommendation algorithm to the metadata string to generate a weighted metric profile for evaluating the GenAI output, wherein the weighted metric profile comprises one or more criteria for evaluation and associated evaluation rules;
• generating, by the processor, an embedding corresponding to the GenAI output, wherein generating the embedding comprises applying a byte-pair encoding model to text outputs, applying a vision-transformer CLS head to image outputs, and applying a pretrained audio embedding model to audio outputs;
• determining, by the processor, weights of the one or more criteria based on similarity between the embedding and domain vectors stored in the knowledge base;
• applying, by the processor, a neural network, trained on benchmark and enterprise-annotated data, to generate per-criterion scores and pass-fail rationales;
• detecting, by the processor, a set of error stubs by comparing the per-criterion scores to corresponding thresholds to identify one or more criterion breaches, wherein the set of error stubs comprising a criterion identifier, a breached threshold, an actual score, a rationale pointer, and one of (i) token byte-offsets for text, (ii) mask coordinates for images, or (iii) time-window indices for audio or video, and wherein the set of error stubs are stored in a structured record;
• localizing, by the processor, one or more error positions within the output based on the set of error stubs, wherein localizing comprises projecting attention maps onto tokens and grouping contiguous token spans for text, selecting image masks having maximum rule-violation gradients for images, or identifying time windows exceeding a violation-likelihood threshold for audio or video, and, responsive to the localization, (a) halting a generation process or automatically requesting regeneration upon a severe criterion breach, and (b) prompting an inference engine to perform targeted re-evaluation limited to the localized token spans, masks, or time windows while preserving non-localized portions of the output;
• and
• updating, by the processor, the knowledge base based on the detected error stubs to continuously improve GenAI output evaluation process, wherein the knowledge base is updated by one or more of adding newly detected error patterns,
• adjusting weights of domain-specific evaluation rules;
• and
• revising threshold for subsequent evaluations, the revised thresholds and weights being persisted in the knowledge base and, via an application interface or software development kit integrated with a deployed Al application, being automatically applied during subsequent inferences of the AI application.