US 2025/0094866 A1 · RESPONDING TO HALLUCINATIONS IN GENERATIVE LARGE LANGUAGE MODELS

Abstract

Techniques for correcting hallucinations produced by generative large language models (LLMs). In one technique, a computing system accesses first output generated by an LLM. The computing system identifies, within the first output, a plurality of assertions. The computing system determines that a first assertion in the plurality of assertions is false. The computing system generates a prompt that indicates that the first assertion is false. The computing system submits the prompt as input to the LLM. The computing system accesses second output that is generated by the LLM, where the second output includes a second assertion that is different than the first assertion and corresponds to the first assertion.

Claims20

Dependent claims

A method comprising:

accessing, by a computing system, first output generated by a large language model (LLM);
identifying, by the computing system, within the first output, a plurality of assertions;
determining, by the computing system, that a first assertion in the plurality of assertions is false;
generating, by the computing system, a prompt that indicates that the first assertion is false;
submitting, by the computing system, the prompt as input to the LLM;
accessing, by the computing system, second output generated by the LLM, wherein the second output includes a second assertion that is different than the first assertion and corresponds to the first assertion;
wherein the method is performed by one or more computing devices.

4 more elements

One or more non-transitory storage media storing instructions which, when executed by one or more computing devices, cause:

accessing, by a computing system, first output generated by a large language model (LLM);
identifying, by the computing system, within the first output, a plurality of assertions;
determining, by the computing system, that a first assertion in the plurality of assertions is false;
generating, by the computing system, a prompt that indicates that the first assertion is false;
submitting, by the computing system, the prompt as input to the LLM;
accessing, by the computing system, second output generated by the LLM, wherein the second output includes a second assertion that is different than the first assertion and corresponds to the first assertion.

3 more elements

Dependencies are inferred from the claim text.

Responding to hallucinations in generative large language models

Life of this patent

Responding to hallucinations in generative large language models

Life of this patent

Abstract

Claims20