A method for AI-powered document retrieval with hallucination prevention, the method comprising:
- receiving, by a generative AI (GEN-AI) agent executing on one or more processors coupled to a memory of an artificial intelligence (AI) model, a regulatory query from a client computing device associated with an organization, the regulatory query comprising a question related to a regulation relevant to an industry of the organization, the GEN-AI agent comprising specialized components including an index evaluator, a document retriever, a document grader, a web searcher, and an answer generator;
- determining, by the index evaluator, whether the regulatory query requires new data retrieval from a web search or whether existing internal data accessible to the AI model can answer the regulatory query;
- when the new data retrieval is required, activating, by the web searcher, the web search utilizing an external large language model (LLM) to identify and retrieve a regulatory document that answers the regulatory query, the utilizing comprising transmitting a structured query request from the one or more processors to the external LLM via a network interface;
- retrieving, by the document retriever, the regulatory document in answer to the regulatory query from at least one of the existing internal data or from the external LLM;
- evaluating, by the document grader, the retrieved regulatory document for regulatory relevance to the regulatory query, the evaluating comprising:
- (i) automatically cross-referencing the retrieved regulatory document against multiple independent online sources to confirm factual accuracy and identify inconsistencies indicative of hallucinated content, the cross-referencing comprising executing comparison algorithms stored in the memory to detect textual inconsistencies between the retrieved regulatory document and the multiple independent online sources, thereby computationally identifying hallucinated content through automated discrepancy detection and thereby improving accuracy of computer-generated regulatory responses;
- and
- (ii) running a supplemental web search to verify that the retrieved regulatory document is relevant and accurate and is not a hallucination;
- generating, by the answer generator, a generated answer to the regulatory query based on the retrieved regulatory document, the generating comprising implementing a multi-tiered validation framework comprising:
- (i) a first AI agent using a first model generating a candidate answer and producing a first output;
- (ii) a second AI agent using a second model architecturally distinct from the first model independently validating the first output by performing parallel analysis of the retrieved regulatory document to produce a second output;
- (iii) automatically comparing the first output and the second output to identify semantic discrepancies, the identification of semantic discrepancies exceeding a predetermined threshold stored in the memory indicating hallucinated content, the predetermined threshold comprising a numerical similarity score value and discrepancies exceeding the predetermined threshold triggering automated re-processing of the regulatory query, thereby reducing hallucinations in computer-generated regulatory answers through computational model disagreement analysis;
- (iv) prompting the external LLM with follow-up verification questions to check veracity of the generated answer and reduce chances of hallucination by leveraging generative capabilities of the external LLM;
- and
- (v) utilizing a vertically-trained artificial intelligence model, specialized in the organization's specific regulatory domain, to establish context of the retrieved regulatory document and reduce chances that hallucinated content from the external LLM will be incorporated into the generated answer;
- and
- transmitting from the one or more processors to the client computing device via the network interface, the generated answer including regulatory content from the retrieved regulatory document and a reference to the retrieved regulatory document.