# US 12,602,541 B2 · Minimizing large language model hallucinations in generated summaries

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,602,541 B2
- Document ID: US12602541B2
- Status: Granted
- Kind: B2
- Application number: 18/644,842
- Filed: 2024-04-24
- Published: 2025-10-30
- Granted: 2026-04-14
- Earliest claimed: 2024-04-24
- Assignees: eBay Inc.
- Applicants: eBay Inc.
- Inventors: Haggai Roitman
- Web page: https://patentgrep.com/patent/US12602541B2

## CPC

- G06F 40/20: Natural language analysis (semantic analysis of natural language G06F40/30)
- G06F 40/56: Natural language generation
- G06F 16/345: {Summarisation for human users}
- G06F 40/30: Semantic analysis
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Systems and methods are directed to minimizing hallucinations in a generated summary. A summary generation system embodied within a server triggers a large language model (LLM) to generate an initial summary for a subject. Based on the initial summary, the server prompts the LLM to generate a list of factual questions about the initial summary. The server then triggers the LLM to answer the list of factual questions without knowledge of the initial summary and using internal knowledge of the LLM. Questions from the list of factual questions that received a positive answer are identified. Based on the questions, the server prompts the LLM to generate a refined summary from the initial summary. The server then generates a user interface that presents the refined summary. Approval of the refined summary triggers generation of a publication using the refined summary.

## Claims

### Claim 1

A method for minimizing hallucinations in generated summaries, the method comprising:
triggering, by a server, a large language model (LLM), to generate an initial summary for a subject;
based on the initial summary, prompting, by the server, the LLM to generate a list of factual questions about the initial summary;
triggering, by the server, the LLM to answer the list of factual questions without knowledge of the initial summary and using internal knowledge of the LLM;
identifying, by the server, questions from the list of factual questions that received a positive answer;
prompting, by the server, the LLM to generate a refined summary from the initial summary based on portions that are relevant to the identified questions; and
generating a user interface that presents the refined summary.

### Claim 2 (depends on 1)

The method of claim 1, wherein the list of factual questions comprises a list of factual polar questions.

### Claim 3 (depends on 1)

The method of claim 1, wherein identifying the questions further comprises filtering out one or more negative questions from the list of factual questions having a negative or unknown answer.

### Claim 4 (depends on 1)

The method of claim 1, wherein triggering generation of the initial summary comprises:
receiving an input including a category and a list of aspects for the subject;
integrating the category into a prompt; and
transmitting the prompt to the LLM, the prompt causing the LLM to generate a list of features that are important for the category.

### Claim 5 (depends on 4)

The method of claim 4, wherein triggering generation of the initial summary further comprises:
integrating the list of aspects into a second prompt; and
based on the second prompt, triggering the LLM to match the list of features with the list of aspects for the subject, the LLM generating the initial summary based on one or more aspects from the list of aspects that match a feature from the list of features.

### Claim 6 (depends on 5)

The method of claim 5, wherein triggering generation of the initial summary comprises:
prompting the LLM to provide a tip with respect to a first feature from the list of features that matches an aspect from the list of aspects, wherein the initial summary emphasizes the first feature and utilizes the tip.

### Claim 7 (depends on 4)

The method of claim 4, wherein the input further comprises a title associated with the subject, the initial summary being based, in part, on the title.

### Claim 8 (depends on 1)

The method of claim 1, wherein the LLM is a closed-source LLM.

### Claim 9 (depends on 1)

The method of claim 1, wherein the user interface further comprises an image of the subject, a list of aspects for the subject, and a selection to approve the refined summary.

### Claim 10 (depends on 1)

The method of claim 1, further comprising:
receiving an indication of approval of the refined summary; and
in response to receiving the indication of approval, generating a publication that includes the refined summary and posting the publication to a network environment.

### Claim 11 (depends on 10)

The method of claim 10, wherein the subject is an item and the refined summary is a part of an item listing.

### Claim 12

A system comprising:
one or more processors; and
a memory storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
triggering, by a server, a large language model (LLM), to generate an initial summary for a subject;
based on the initial summary, prompting, by the server, the LLM to generate a list of factual questions about the initial summary;
triggering, by the server, the LLM to answer the list of factual questions without knowledge of the initial summary and using internal knowledge of the LLM;
identifying, by the server, questions from the list of factual questions that received a positive answer;
prompting, by the server, the LLM to generate a refined summary from the initial summary based on portions that are relevant to the identified questions; and
generating a user interface that presents the refined summary.

### Claim 13 (depends on 12)

The system of claim 12, wherein the list of factual questions comprises a list of factual polar questions.

### Claim 14 (depends on 12)

The system of claim 12, wherein identifying the questions further comprises filtering out one or more negative questions from the list of factual questions having a negative or unknown answer.

### Claim 15 (depends on 12)

The system of claim 12, wherein triggering generation of the initial summary comprises:
receiving an input including a category and a list of aspects for the subject;
integrating the category into a prompt; and
transmitting the prompt to the LLM, the prompt causing the LLM to generate a list of features that are important for the category.

### Claim 16 (depends on 15)

The system of claim 15, wherein triggering generation of the initial summary further comprises:
integrating the list of aspects into a second prompt; and
based on the second prompt, triggering the LLM to match the list of features with the list of aspects for the subject, the LLM generating the initial summary based on one or more aspects from the list of aspects that match a feature from the list of features.

### Claim 17 (depends on 16)

The system of claim 16, wherein triggering generation of the initial summary comprises:
prompting the LLM to provide a tip with respect to a first feature from the list of features that matches an aspect from the list of aspects, wherein the initial summary emphasizes the first feature and utilizes the tip.

### Claim 18 (depends on 15)

The system of claim 15, wherein the input further comprises a title associated with the subject, the initial summary being based, in part, on the title.

### Claim 19 (depends on 12)

The system of claim 12, wherein the operations further comprise:
receiving an indication of approval of the refined summary; and
in response to receiving the indication of approval, generating a publication that includes the refined summary and posting the publication to a network environment.

### Claim 20

A machine-storage medium comprising instructions which, when executed by one or more processors of a machine, cause the machine to perform operations comprising:
triggering, by a server, a large language model (LLM), to generate an initial summary for a subject;
based on the initial summary, prompting, by the server, the LLM to generate a list of factual questions about the initial summary;
triggering, by the server, the LLM to answer the list of factual questions without knowledge of the initial summary and using internal knowledge of the LLM;
identifying, by the server, questions from the list of factual questions that received a positive answer;
prompting, by the server, the LLM to generate a refined summary from the initial summary based on portions that are relevant to the identified questions; and
generating a user interface that presents the refined summary.