Retrieval augmented generation
AssigneeIntuit Inc.
InventorsJolene Huang, Pankaj Rastogi, Spriha Awasthi, Yiran Huang
Life of this patent
Estimating the term…
Abstract
Certain aspects of the present disclosure provide techniques retrieval augmented generation of language model responses using an embedding database. Embeddings for data is stored in an embedding database. When a prompt related to the data is received, relevant embeddings may be retrieved from the database and used generate an augmented prompt based on the initial prompt and the retrieved embeddings from the database. The augmented prompt can be input into a machine learning model. Although the model may be unaware of the data from which the embeddings of the embedding database were generated, the augmented prompt enables the model to use the data to improve breadth and depth of responses.
Claims18
A method, comprising:
- receiving a prompt from a user at an endpoint;
- preprocessing the prompt by:
- performing summarization on the prompt;
- performing entity extraction on the prompt;
- and
- performing classification on the prompt;
- providing the prompt to an embedding module to generate an embedding for the prompt;
- retrieving one or more embeddings from a vector database based on the embedding for the prompt and based on a result of the preprocessing of the prompt;
- generating an augmented prompt based on the one or more embeddings and the embedding for the prompt;
- generating a response to the augmented prompt using a machine learning model and using a context of the one or more embeddings as a context for the machine learning model;
- and
- providing the response to the user at the endpoint.
A system comprising:
- a memory having executable instructions stored thereon;
- an endpoint having a user interface associated with a display;
- and
- one or more processors configured to execute the executable instructions to cause the system to perform a method comprising:
- receiving a prompt from a user at the user interface;
- providing the prompt to an embedding module to generate an embedding for the prompt;
- retrieving one or more embeddings from a vector database based on the embedding for the prompt, wherein the retrieving the one or more embeddings comprises performing a semantic search using a similarity algorithm and the embedding for the prompt to identify one or more similar embeddings in the vector database, the similar embeddings being similar to the embedding for the prompt;
- generating an augmented prompt based on the one or more embeddings and the embedding for the prompt, wherein the generating of the augmented prompt comprises combining the similar embeddings and the embedding for the prompt to create an augmented embedding;
- generating a response to the augmented prompt using a machine learning model and using a context of the one or more embeddings as a context for the machine learning model;
- and
- providing the response to the user on the display.
A non-transitory computer readable storage medium comprising instructions, that when executed by one or more processors of a computing system, cause the computing system to perform a method comprising:
- receiving a corpus of data;
- determining a segment of the corpus of data corresponds to a conversation;
- determining a context of the conversation;
- chunking the corpus of data to produce a plurality of data chunks, wherein the chunking of the corpus of data comprises generating a data chunk corresponding to the segment;
- generating a plurality of embeddings for the plurality of data chunks, wherein the generating of the plurality of embeddings comprises generating an embedding for the segment that includes metadata defining the context of the conversation as a context for the embedding;
- storing the plurality of embeddings in a vector database;
- receiving a prompt from a user at an endpoint;
- providing the prompt to an embedding module to generate an embedding for the prompt;
- retrieving one or more embeddings from the vector database based on the embedding for the prompt;
- generating an augmented prompt based on the one or more embeddings and the embedding for the prompt;
- generating a response to the augmented prompt using a machine learning model and using a context of the one or more embeddings as a context for the machine learning model;
- and
- providing the response to the user at the endpoint.
Dependencies are inferred from the claim text.