# US 2026/0073295 A1 · Generative response engine using chain-of-thought reasoning

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0073295 A1
- Document ID: US20260073295A1
- Status: Published application
- Kind: A1
- Application number: 19/170,839
- Filed: 2025-04-04
- Published: 2026-03-12
- Earliest claimed: 2024-09-11
- Assignees: OpenAI OpCo, LLC
- Applicants: OpenAI OpCo, LLC.
- Inventors: Peter Vidani; Valerie Qi
- Web page: https://patentgrep.com/patent/US20260073295A1

## CPC

- G06N 20/00: Machine learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

The present technology pertains to a generative response system (system) that includes a chain-of-thought (CoT) reasoning model. The system receives a prompt for a response, wherein the response benefits from multi-step, CoT reasoning. The prompt is tokenized to generate input tokens, which can also include tokens representing a contextual conversation history. A first machine learning (ML) model having a CoT functionality processes the input tokens, generating reasoning tokens, which explore one or more reasoning frameworks for responding to the prompt. The combination of the first and second tokens is processed to generate output tokens representing the response sent to the requester. The second tokens are not provided to the requester and are omitted from the chat history. However, a summary of the multi-step reasoning framework used to generate the response can be generated based on the second tokens and presented to the requester.

## Claims

### Claim 1

A method of performing chain-of-thought (CoT) reasoning using one or more machine learning (ML) models, the method comprising:
receiving, from a requester, a prompt that is a request for a response, wherein the response benefits from multi-step reasoning:
tokenizing the prompt to generate prompt tokens, wherein first tokens include the prompt tokens;
processing, by a first ML model, the first tokens to generate second tokens, the second tokens that explore one or more reasoning frameworks for responding to the prompt;
processing a combination of the first tokens and the second tokens to generate third tokens representing the response; and
providing, to the requester, the response without information representing the second tokens.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
processing the second tokens to generate a summary of an applied framework of the one or more reasoning frameworks, wherein
the applied framework is a framework of the one or more reasoning frameworks that is applied to generate the third tokens,
the summary represents a description of the applied framework, and
the response represents a reply to the prompt that is generated using the applied framework.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
causing the response together with the summary to be presented in a user interface, wherein
a presentation of the summary in the user interface is configured to be collapsed and expanded, and the user interface includes a time representing a period over which the second tokens were generated.

### Claim 4 (depends on 3)

The method of claim 3, wherein:
the user interface further presents a time value representing a period over which the second tokens were generated, and
the summary includes respective titles and respective descriptions corresponding to steps within the applied framework.

### Claim 5 (depends on 2)

The method of claim 2, wherein:
the summary is presented inline with the response, or
the summary is presented in a panel offset from a presentation of the response.

### Claim 6 (depends on 1)

The method of claim 1, wherein processing the combination of the first tokens and the second tokens to generate the third tokens further includes, after a period during which the first tokens are processed to generate the second tokens, streaming the response to the requester as the third tokens are generated using an autoregressive ML method.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
determining, by the first ML model, chunks of the second tokens that represent steps of an applied framework of the one or more reasoning frameworks, and
processing the chunks of the second tokens to generate step summaries of the respective steps as the chunks of the second tokens are being determined, such that a step summary of a first step of the steps is generated based on a first chunk before a second chunk corresponding to a second step has been determined, wherein
a summary of the applied framework comprises the step summaries.

### Claim 8 (depends on 7)

The method of claim 7, wherein the step summary of the first step is generated before generation of the second tokens is complete.

### Claim 9 (depends on 7)

The method of claim 7, wherein a conversation thread resulting from generation of the second tokens, the third tokens and the summary comprises information of the first tokens and the third tokens but lacks information of the second tokens and the summary.

### Claim 10 (depends on 7)

The method of claim 7, wherein:
a second ML model processes the chunks of the second tokens to generate the step summaries,
the first ML model uses a chain-of-thought functionality to develop a multi-step framework for responding to the prompt, and
the second ML model is a language model that lacks the chain-of-thought functionality.

### Claim 11 (depends on 10)

The method of claim 10, wherein the first ML model and the second ML model respectively comprise autoregressive ML models that generate one token at a time in response to previous tokens comprising input tokens and previously generated output tokens.

### Claim 12 (depends on 1)

The method of claim 1, wherein the summary describes steps of an applied framework used to generate the response based on the second tokens, and the summary suggests a path for verifying the response and the applied framework.

### Claim 13 (depends on 1)

The method of claim 1, wherein the requester is an application programming interface (API).

### Claim 14

A computing apparatus comprising:
a processor; and
a memory storing instructions that, when executed by the processor, configure the apparatus to perform operations:
receiving, from a requester, a prompt that is a request for a response, wherein the response benefits from multi-step reasoning:
tokenizing the prompt to generate prompt tokens, wherein first tokens include the prompt tokens;
processing, by a first ML model, the first tokens to generate second tokens, the second tokens that explore one or more reasoning frameworks for responding to the prompt;
processing a combination of the first tokens and the second tokens to generate third tokens representing the response; and
providing, to the requester, the response without information representing the second tokens.

### Claim 15 (depends on 14)

The computing apparatus of claim 14, wherein the instructions further configure the apparatus to perform operations:
processing the second tokens to generate a summary of an applied framework of the one or more reasoning frameworks, the applied framework being a framework of the one or more reasoning frameworks that is applied to generate the third tokens; and
causing the response together with the summary to be presented in a user interface, wherein
the summary represents a description of the applied framework, and
the response represents a reply to the prompt that is generated using the applied framework.

### Claim 16 (depends on 14)

The computing apparatus of claim 14, wherein the instructions further configure the apparatus to perform operations:
determining, by the first ML model, chunks of the second tokens that represent steps of an applied framework of the one or more reasoning frameworks, and
processing the chunks of the second tokens to generate step summaries of the respective steps as the chunks of the second tokens are being determined, such that a step summary of a first step of the steps is generated based on a first chunk before a second chunk corresponding to a second step has been determined, wherein
a summary of the applied framework comprises the step summaries.

### Claim 17 (depends on 16)

The computing apparatus of claim 16, wherein:
a second ML model processes the chunks of the second tokens to generate the step summaries,
the first ML model uses a chain-of-thought functionality to develop a multi-step framework for responding to the prompt, and
the second ML model is a language model that lacks the chain-of-thought functionality.

### Claim 18 (depends on 16)

The computing apparatus of claim 16, wherein a conversation thread resulting from generation of the second tokens, the third tokens and the summary comprises information of the first tokens and the third tokens but lacks information of the second tokens and the summary.

### Claim 19 (depends on 14)

The computing apparatus of claim 14, wherein a conversation thread resulting from generation of the second tokens and the third tokens comprises information of the first tokens and the third tokens but lacks information of the second tokens.

### Claim 20

A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to:
receive, from a requester, a prompt that is a request for a response, wherein the response benefits from multi-step reasoning:
tokenizing the prompt to generate prompt tokens, wherein first tokens include the prompt tokens;
process, by a first ML model, the first tokens to generate second tokens, the second tokens that explore one or more reasoning frameworks for responding to the prompt;
process a combination of the first tokens and the second tokens to generate third tokens representing the response; and
provide, to the requester, the response without information representing the second tokens.
