Response latency in generative models
AssigneeGoogle Inc.
InventorClark Alan DuVall
Life of this patent
Estimating the term…
Abstract
Disclosed implementations use generative machine learning methods to generate an internal state based on content from a resource and provide responses to prompts based on the internal state. In an example implementation, content from a resource is converted to tokens based on a vocabulary associated with a task. A state for a generative model is generated by processing the tokens through the generative model. A prompt from a user interface and the prompt is processed through the generative model to generate a response based on the state. The response is provided to the user interface.
Claims20
A method comprising:
- converting content from a resource to tokens;
- generating a state for a generative model by processing the tokens through the generative model;
- receiving a prompt from a user interface;
- processing the prompt through the generative model to generate a response based on the state;
- and
- providing the response to the user interface.
A non-transitory computer-readable medium storing executable instructions that when executed an electronic processor, cause the electronic processor to:
- convert content from a resource to tokens;
- generate a state for a generative model by processing the tokens through the generative model;
- receive a prompt from a user interface;
- process the prompt through the generative model to generate a response based on the state;
- and
- provide the response to the user interface.
A system comprising:
- a user interface;
- a generative model;
- and
- an electronic processor communicably coupled to the user interface and configured to:
- convert content from a resource to tokens based on a vocabulary associated with a task selected via the user interface;
- generate a state for the generative model by processing the tokens through the generative model;
- receive a prompt from the user interface;
- process the prompt through the generative model to generate a response based on the state;
- and
- provide the response to the user interface.
Dependencies are inferred from the claim text.