# US 2026/0064781 A1 · Llm framework for large scale applications

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0064781 A1
- Document ID: US20260064781A1
- Status: Published application
- Kind: A1
- Application number: 19/313,108
- Filed: 2025-08-28
- Published: 2026-03-05
- Earliest claimed: 2024-08-29
- Assignees: DoorDash, Inc.
- Applicants: DoorDash, Inc.
- Inventors: Zhe Jia; Shuai Wang; Aditi Bamba; Yu Wang
- Web page: https://patentgrep.com/patent/US20260064781A1

## CPC

- G06F 16/93: Document management systems
- G06F 16/215: Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
- G06F 16/24578: {using ranking}
- G06F 16/3329: Natural language query formulation
- G06N 3/02: Neural networks
- G06N 3/0475: Generative networks
- G06N 20/00: Machine learning
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A method includes a computer receiving a user query. The computer generates a summary of the user query using a first large language model. The computer determines a user issue from a first database based on the summary. The computer determines digital document from a second database based on the user issue. The computer generates a prompt based on the digital document and a prompt template. The computer generates a response based on the prompt using a second large language model.

## Claims

### Claim 1

A method comprising:
receiving, by a computer, a user query;
generating, by the computer, a summary of the user query using a first large language model;
determining, by the computer, a user issue from a first database based on the summary;
determining, by the computer, a digital document from a second database based on the user issue;
generating, by the computer, a prompt based on the digital document and a prompt template; and
generating, by the computer, a response based on the prompt using a second large language model.

### Claim 2 (depends on 1)

The method of claim 1, wherein the user query includes a chat history between a user and a chatbot.

### Claim 3 (depends on 1)

The method of claim 1, wherein after generating the response, the method further comprises:
evaluating, by the computer, quality of the response.

### Claim 4 (depends on 3)

The method of claim 3, wherein evaluating the quality of the response comprises:
generating, by the computer, a digital document segment embedding based on the digital document;
generating, by the computer, a response embedding based on the response; and
determining, by the computer, a similarity score by comparing the digital document segment embedding with the response embedding.

### Claim 5 (depends on 4)

The method of claim 4, wherein evaluating the quality of the response further comprises:
if the similarity score is below a similarity score threshold, obtaining, by the computer, a guardrail prompt template;
generating, by the computer, a guardrail prompt using the guardrail prompt template; and
regenerating, by the computer, the response based on the guardrail prompt using the second large language model.

### Claim 6 (depends on 1)

The method of claim 1, wherein the user query is received from a user device and wherein the method further comprises:
providing, by the computer, the response to the user device.

### Claim 7 (depends on 1)

The method of claim 1, wherein generating the summary comprises:
generating, by the computer, a summarization prompt using a summarization prompt template and the user query;
inputting, by the computer, the summarization prompt into the first large language model; and
obtaining, by the computer, the summary as output from the first large language model.

### Claim 8 (depends on 1)

The method of claim 1, wherein determining the user issue comprises:
generating, by the computer, an issue request message comprising the summary;
providing, by the computer, the issue request message to a third database, wherein the third database obtains a plurality of issues that are similar to the summary, generates an issue response message comprising the plurality of issues, and provides the issue response message to the computer;
receiving, by the computer, the issue response message; and
selecting, by the computer, an issue of the plurality of issues to be the user issue.

### Claim 9 (depends on 1)

The method of claim 1, wherein determining the digital document comprises:
generating, by the computer, a digital document identifier request message comprising the user issue;
providing, by the computer, the digital document identifier request message to a third database, wherein the third database obtains a digital document identifier that is stored in association with the user issue, generates a digital document identifier response message comprising the digital document identifier, and provides the digital document identifier response message to the computer;
receiving, by the computer, the digital document identifier response message;
generating, by the computer, a digital document request message comprising the digital document identifier providing, by the computer, the digital document request message to the second database, wherein the second database identifies the digital document using the digital document identifier, generates a digital document response message comprising the digital document, and provides the digital document response message to the computer; and
receiving, by the computer, the digital document response message comprising the digital document.

### Claim 10 (depends on 1)

The method of claim 1, further comprising:
selecting, by the computer, the prompt template from a plurality of prompt templates stored in a prompt template database.

### Claim 11 (depends on 1)

The method of claim 1, wherein the prompt is a first prompt, wherein the method further comprises:
obtaining, by the computer, an open ended review document, a judge template, a historical transcript, and historical data comprising the response and the user query;
generating, by the computer, a second prompt using the open ended review document, the judge template, the historical transcript, and the historical data;
generating, by the computer, an output using a third large language model and the second prompt; and
storing, by the computer, the output into an open ended results database, wherein the output is analyzed and summarized by an analysis and summarization module to improve performance of the computer, the first large language model, and/or the second large language model.

### Claim 12

A computer comprising:
a processor; and
a non-transitory computer readable medium comprising code, executable by the processor for performing a method comprising:
receiving a user query;
generating a summary of the user query using a first large language model;
determining a user issue from a first database based on the summary;
determining a digital document from a second database based on the user issue;
generating a prompt based on the digital document and a prompt template; and
generating a response based on the prompt using a second large language model.

### Claim 13 (depends on 12)

The computer of claim 12, wherein the method further comprises:
obtaining a digital document segment embedding that is associated with the digital document, from a third database;
generating a response embedding based on the response;
determining a similarity score by comparing the digital document segment embedding with the response embedding; and
comparing the similarity score to a similarity score threshold to determine quality of the response.

### Claim 14 (depends on 13)

The computer of claim 13, wherein if the similarity score is less than the similarity score threshold, the method further comprises:
obtaining a guardrail prompt template;
generating a guardrail prompt using guardrail prompt template, the digital document, and the user issue; and
regenerating the response based on the guardrail prompt using the second large language model.

### Claim 15 (depends on 12)

The computer of claim 12, wherein the prompt is a first prompt, wherein the method further comprises:
obtaining a structured review document, a judge template, a historical transcript, and historical data comprising the response and the user query;
generating a second prompt using the structured review document, the judge template, the historical transcript, and the historical data;
generating an output using a third large language model and the second prompt; and
storing the output into a structured results database, wherein the output is provided to a dashboard.

### Claim 16 (depends on 12)

The computer of claim 12, wherein determining the user issue comprises:
generating an issue request message comprising the summary;
providing the issue request message to a third database, wherein the third database obtains a plurality of issues that are similar to the summary, generates an issue response message comprising the plurality of issues, and provides the issue response message to the computer;
receiving the issue response message; and
selecting an issue of the plurality of issues to be the user issue.

### Claim 17 (depends on 12)

The computer of claim 12, wherein the user query includes a text question.

### Claim 18

A method comprising:
displaying, by a user device, a text chat between a user and a chatbot hosted by a computer;
receiving as input, by the user device, one or more text messages from the user for the text chat;
providing, by the user device, the one or more text messages to the computer, wherein the one or more text messages and other messages from the text chat are included in a user query, wherein the computer generates a summary of the user query using a first large language model, determines a user issue from a first database based on the summary, determines a digital document from a second database based on the user issue, generates a prompt based on the digital document and a prompt template, and generates a response based on the prompt using a second large language model; and
receiving, by the user device, the response from the computer.

### Claim 19 (depends on 18)

The method of claim 18, wherein the response includes a link to the digital document.

### Claim 20 (depends on 19)

The method of claim 19, wherein the digital document is an article.
