# US 2026/0228500 A1 · HALLUCINATION DETECTION SYSTEM FOR LARGE LANGUAGE MODELS (LLMs)

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0228500 A1
- Document ID: US20260228500A1
- Status: Published application
- Kind: A1
- Application number: 19/047,368
- Filed: 2025-02-06
- Published: 2026-08-06
- Earliest claimed: 2025-02-06
- Inventors: Richard Gardner; John Jozwiak
- Web page: https://patentgrep.com/patent/US20260228500A1

## CPC

- G06N 3/047: Probabilistic or stochastic networks
- G06N 20/00: Machine learning
- G06N 3/006: based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
- G06N 3/042: Knowledge-based neural networks; Logical representations of neural networks
- G06N 3/043: based on fuzzy logic, fuzzy membership or fuzzy inference, e.g. adaptive neuro-fuzzy inference systems [ANFIS]
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/045: Combinations of networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0475: Generative networks
- G06N 3/08: Learning methods
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06N 3/09: Supervised learning
- G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
- G06N 5/02: Knowledge representation; Symbolic representation
- G06N 5/022: Knowledge engineering; Knowledge acquisition
- G06N 5/04: Inference or reasoning models
- G06N 5/041: {Abduction}
- G06N 5/045: Explanation of inference; Explainable artificial intelligence [XAI]; Interpretable artificial intelligence
- G06N 20/10: using kernel methods, e.g. support vector machines [SVM]
- G06N 20/20: Ensemble learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A method of detecting hallucinations by large language models (LLMs) is disclosed. A first prompt is received. The first prompt is submitted to an LLM. A first response is received from the LLM. The first response is generated by the LLM in response to the first prompt. A second prompt is generated. The second prompt is designed to detect a probability that the first response includes a hallucination. The second prompt is submitted to the LLM or an additional LLM. A second response is received. The second response is generated by the LLM or the additional LLM in response to the second prompt. The second response is analyzed to perform the detecting of the probability that the first response includes the hallucination. A third response is generated based on the analyzing of the second response. The third response is provided in response to the first prompt.

## Claims

### Claim 1

A system comprising:
one or more computer processors;
one or more computer memories;
a set of instructions stored in the one or more computer memories, the set of instructions configuring the one or more computer processors to perform operations, the operations comprising:
receiving or generating a first prompt;
submitting the first prompt to a large language model (LLM);
receiving a first response from the large language model;
generating a second prompt, the second prompt explicitly requiring the LLM to operate under an assumption that the first response is incorrect or requires reexamination by including a statement that presupposes the first response is incorrect or requires reexamination, the statement referencing some or all of the first response or a variation of the first response and asserting that the first response the some or all of the first response or the variation of the first response is incorrect or requires re-examination;
submitting the second prompt to the LLM or an additional LLM;
receiving a second response from the LLM or the additional LLM;
determining, based on the second response, whether the first response is likely valid or invalid, the determining of whether the first response is likely valid or invalid including applying one or more verification algorithms to the second response, the one or more verification algorithms selected by one or more artificial intelligence (AI) models based on one or more of a content type of the first prompt or the first response, a domain of the first prompt or the first response, or a type of potential hallucination; and
providing a third response in response to the first prompt, the third response including an indication of the determining of whether the first response is likely valid or invalid or the third response correcting one or more portions of the first response.

### Claim 2 (depends on 1)

The system of claim 1, wherein the providing of the third response includes appending to the first response an indication of a probability that the first response includes a hallucination.

### Claim 3 (depends on 2)

The system of claim 2, wherein the providing of the third response includes generating an identification of the hallucination.

### Claim 4 (depends on 1)

The system of claim 1, wherein the providing of the third response includes generating an indication of the correction.

### Claim 5 (depends on 1)

The system of claim 1, wherein the determining of whether the first response is likely valid or invalid is based on a probability that the first response includes a hallucination transgressing a validity threshold or an invalidity threshold.

### Claim 6 (depends on 1)

The system of claim 1, wherein the generating of the second prompt includes generating a request to explain in detail why the first response is incorrect.

### Claim 7

(canceled)

### Claim 8 (depends on 1)

The system of claim 1, wherein the generating of the statement includes prefixing the first response with text indicating the first response cannot be correct.

### Claim 9 (depends on 1)

The system of claim 1, wherein the determining that the first response is likely valid or invalid is based on the second response not acknowledging the first response was correct.

### Claim 10 (depends on 1)

The system of claim 1, wherein the determining that the first response is likely valid or invalid is based on the second response acknowledging that that the first response was incorrect.

### Claim 11 (depends on 10)

The system of claim 10, the operations further comprising extracting a fourth response from the second response, assigning the fourth response a probability that the fourth response is correct, the probability that the fourth response is correct being greater than a probability that the first response is correct.

### Claim 12 (depends on 1)

The system of claim 1, wherein the generating of the second prompt includes generating a request for the LLM or the additional LLM to identify one or more prompts that would generate the first response the determining of whether the first response is likely valid or invalid is based on whether the first prompt is included in the one or more prompts identified in the second response.

### Claim 13 (depends on 1)

The system of claim 1, wherein the generating of the second prompt comprises requesting the LLM or the additional LLM to think through each step carefully, avoid making assumptions, favor a simple solution, output the best response possible, or output the response in a concise manner.

### Claim 14 (depends on 1)

The system of claim 1, wherein the generating of the second prompt includes generating a request that the LLM or the additional LLM to perform one or more steps of pertaining to the generating of the first response in reverse order.

### Claim 15 (depends on 1)

The system of claim 1, wherein the LLM corresponds to a first instance of the LLM and the additional LLM corresponds to a second instance of the LLM.

### Claim 16 (depends on 1)

The system of claim 1, wherein the additional LLM is trained in a different manner from the LLM.

### Claim 17 (depends on 16)

The system of claim 16, the operations further comprising reconciling the first response with the second response, the reconciling including determining whether the second response acknowledges or refutes the first response.

### Claim 18 (depends on 17)

The system of claim 17, wherein the determining of whether the second response acknowledges or refutes the first response is based on the second response providing or failing to provide an acknowledgement or a refutation with at least a threshold level of detail.

### Claim 19

A method comprising:
receiving or generating a first prompt;
submitting the first prompt to a large language model (LLM);
receiving a first response from the large language model;
generating a second prompt, the second prompt explicitly requiring the LLM to operate under an assumption that the first response is incorrect or requires reexamination by including a statement that presupposes the first response is incorrect or requires reexamination, the statement referencing some or all of the first response or a variation of the first response and asserting that the first response the some or all of the first response or the variation of the first response is incorrect or requires re-examination;
submitting the second prompt to the LLM or an additional LLM;
receiving a second response from the LLM or the additional LLM;
determining, based on the second response, whether the first response is likely valid or invalid, the determining of whether the first response is likely valid or invalid including applying one or more verification algorithms to the second response, the one or more verification algorithms selected by one or more artificial intelligence (AI) models based on one or more of a content type of the first prompt or the first response, a domain of the first prompt or the first response, or a type of potential hallucination; and
providing a third response in response to the first prompt, the third response including an indication of the determining of whether the first response is likely valid or invalid or the third response correcting one or more portions of the first response.

### Claim 20

A non-transitory computer-readable storage medium storing a set of instructions that, when executed by one or more computer processors, causes the one or more computer processors to perform operations, the operations comprising:
receiving or generating a first prompt;
submitting the first prompt to a large language model (LLM);
receiving a first response from the large language model;
generating a second prompt, the second prompt explicitly requiring the LLM to operate under an assumption that the first response is incorrect or requires reexamination by including a statement that presupposes the first response is incorrect or requires reexamination, the statement referencing some or all of the first response or a variation of the first response and asserting that the first response the some or all of the first response or the variation of the first response is incorrect or requires re-examination;
submitting the second prompt to the LLM or an additional LLM;
receiving a second response from the LLM or the additional LLM;
determining, based on the second response, whether the first response is likely valid or invalid, the determining of whether the first response is likely valid or invalid including applying one or more verification algorithms to the second response, the one or more verification algorithms selected by one or more artificial intelligence (AI) models based on one or more of a content type of the first prompt or the first response, a domain of the first prompt or the first response, or a type of potential hallucination; and
providing a third response in response to the first prompt, the third response including an indication of the determining of whether the first response is likely valid or invalid or the third response correcting one or more portions of the first response.
