Minimizing large language model hallucinations in generated summaries

A system comprising:

• one or more processors;
• and
• a memory storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
• triggering, by a server, a large language model (LLM), to generate an initial summary for a subject;
• based on the initial summary, prompting, by the server, the LLM to generate a list of factual questions about the initial summary;
• triggering, by the server, the LLM to answer the list of factual questions without knowledge of the initial summary and using internal knowledge of the LLM;
• identifying, by the server, questions from the list of factual questions that received a positive answer;
• prompting, by the server, the LLM to generate a refined summary from the initial summary based on portions that are relevant to the identified questions;
• and
• generating a user interface that presents the refined summary.

A machine-storage medium comprising instructions which, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

• triggering, by a server, a large language model (LLM), to generate an initial summary for a subject;
• based on the initial summary, prompting, by the server, the LLM to generate a list of factual questions about the initial summary;
• triggering, by the server, the LLM to answer the list of factual questions without knowledge of the initial summary and using internal knowledge of the LLM;
• identifying, by the server, questions from the list of factual questions that received a positive answer;
• prompting, by the server, the LLM to generate a refined summary from the initial summary based on portions that are relevant to the identified questions;
• and
• generating a user interface that presents the refined summary.