Integrated self-evaluation and follow-up suggestion mechanism for rag systems

A system for performing retrieval-augmented generation (RAG), in large language model (LLM) having pre-trained knowledge and additional content for being retrieved from a data management system, said pre-trained knowledge and additional content forming said RAG systems knowledge base said system comprising:

• a RAG system having at least one processor, a memory for storing data and instructions for execution by the at least one processor, an input/output subsystem and the LLM;
• a large language model operatively coupled to the RAG system;
• an external content having a data management system for storing said additional content operatively coupled to the RAG system;
• said RAG system for being operatively coupled to a user computer system, said user computer system for providing a question to the LLM which generates content in response to the question and determines whether additional content needs to be retrieved from a data management system to provide a correct answer in response to the question,
• wherein the AI model is provided with the user question and the AI generated content;
• and
• the RAG system asks the AI model to score its ability to have confidently answered the question, wherein said scoring uses the pre-trained knowledge of the LLM;
• if the score is below a predefined threshold, instructing the AI model to generate an alternate response using said retrieved additional content which alternate response is presented.