Back to results
US 12,591,766 B1Granted
Application programming interface with generative response engine state management
AssigneeOpenAI OpCo, LLC
InventorsSteve Coffey, Nikunj Handa, Manoli Liodakis
Life of this patent
Estimating the term…
Abstract
The present technology pertains to an API for state management and function chaining using a generative response engine. The present technology includes an API which takes a response ID as a parameter. The response ID can be used to retrieve a chain of responses including the previous response and any preceding responses, which is provided to the context window of an artificial intelligence (AI) model of the generative response engine. Additionally, the API can be used to enforce constrained sampling by the AI model to enable tool chaining without repetitive programming.
Claims20
A method comprising:
- receiving, from a client application via an application programming interface (API), a request comprising a prompt for an artificial intelligence (AI) model of a generative response engine and a first response ID, wherein the first response ID was provided by the AI model in a previous response to identify the previous response;
- retrieving, by the generative response engine from a database, a chain of responses in a sequence that ends with the previous response having the first response ID, wherein the chain of responses is part of a tree of responses and wherein the chain of responses constitutes a first state representation;
- passing, to the AI model, the prompt and the first state representation, wherein the first state representation is provided in a context window of the AI model;
- obtaining, by the AI model, a previous context from the previous response based on the first state representation without replicating a conversation leading to the previous response;
- outputting, by the AI model, a response to the prompt and a second response ID corresponding to the response, wherein the response is based at least in part on the previous context;
- and
- receiving, by the API from the generative response engine, the response to the prompt and the second response ID.
4 more elements
A computing system comprising:
- at least one processor;
- and
- a memory storing instructions that, when executed by the at least one processor, configure the computing system to:
- receive, from a client application via an application programming interface (API), a request comprising a prompt for an artificial intelligence (AI) model and a first response ID, wherein the first response ID was provided by the AI model in a previous response to identify the previous response;
- retrieve, from a database, a chain of responses in a sequence that ends with the previous response having the first response ID, wherein the chain of responses is part of a tree of responses and wherein the chain of responses constitutes a first state representation;
- pass, to the AI model, the prompt and the first state representation, wherein the first state representation is provided in a context window of the AI model;
- obtain a previous context from the previous response based on the first state representation without replicating a conversation leading to the previous response;
- output a response to the prompt and a second response ID corresponding to the response, wherein the response is based at least in part on the previous context;
- and
- receive, by the API from the AI model, the response to the prompt and the second response ID.
7 more elements
A non-transitory computer-readable medium comprising instructions that when executed by at least one processor, cause the at least one processor to:
- receive, from a client application via an application programming interface (API), a request comprising a prompt for an artificial intelligence (AI) model and a first response ID, wherein the first response ID was provided by the AI model in a previous response to identify the previous response;
- retrieve, from a database, a chain of responses in a sequence that ends with the previous response having the first response ID, wherein the chain of responses is part of a tree of responses and wherein the chain of responses constitutes a first state representation;
- pass, to the AI model, the prompt and the first state representation, wherein the first state representation is provided in a context window of the AI model;
- obtain a previous context from the previous response based on the first state representation without replicating a conversation leading to the previous response;
- output a response to the prompt and a second response ID corresponding to the response, wherein the response is based at least in part on the previous context;
- and
- receive, by the API from the AI model, the response to the prompt and the second response ID.
4 more elements
Dependencies are inferred from the claim text.