US 2026/0228288 A1 · HYBRID RETRIEVAL AUGMENTED GENERATION FOR RICH DOCUMENT QUERIES USING A LARGE LANGUAGE MODEL

Abstract

Techniques for implementing hybrid retrieval augmented generation for rich document queries using a large language model (“LLM”) are disclosed. In an example method, a computing system receives a query about documents in a vector database including terms. The computing system generates tokens based on the terms. The computing system determines a ranking of the documents using a probabilistic ranking method based on the tokens. The computing system outputs, to the vector database, a query to cause a vector similarity search based on an embedded representation of the query. The computing system receives, from the vector database, a ranking of the documents. The computing system determines top documents based on the ranking and the ranking. The computing system determines a knowledge graph based on the top documents. The computing system receives, from an LLM, a response to a prompt including the query, the top documents, and the knowledge graph.

Claims20

Dependent claims

A method, comprising:

receiving, from a client device, a first query about one or more documents in a vector database including one or more first terms;
generating one or more first tokens based on the one or more first terms;
determining a first ranking of the one or more documents using a probabilistic ranking method based on the one or more first tokens;
outputting, to the vector database, a first vector database query to cause a first vector similarity search based on a first embedded representation of the first query;
receiving, from the vector database, a second ranking of the one or more documents in response to the first vector database query;
determining one or more first top documents based on the first ranking and the second ranking;
determining a first knowledge graph based on the one or more first top documents;
receiving, from a first large language model (“LLM”), a first response to a prompt including the first query, the one or more first top documents, and at least a portion of the first knowledge graph;
and
outputting the first response to the client device.

7 more elements

A non-transitory computer-readable storage medium storing processor-executable instructions configured to cause one or more processors to:

receive, from a client device, a first query about one or more documents in a vector database including one or more first terms;
generate one or more first tokens based on the one or more first terms;
determine a first ranking of the one or more documents using a probabilistic ranking method based on the one or more first tokens;
output, to the vector database, a first vector database query to cause a first vector similarity search based on a first embedded representation of the first query;
receive, from the vector database, a second ranking of the one or more documents in response to the first vector database query;
determine one or more first top documents based on the first ranking and the second ranking;
determine a first knowledge graph based on the one or more first top documents;
receive, from a first large LLM, a first response to a prompt including the first query, the one or more first top documents, and at least a portion of the first knowledge graph;
and
output the first response to the client device.

7 more elements

A system comprising:

one or more non-transitory computer-readable media;
and
one or more processors communicatively coupled to the one or more non-transitory computer-readable media, the one or more processors configured to execute processor-executable instructions stored in the non-transitory computer-readable media to:
receive, from a client device, a first query about one or more documents in a vector database including one or more first terms;
generate one or more first tokens based on the one or more first terms;
determine a first ranking of the one or more documents using a probabilistic ranking method based on the one or more first tokens;
output, to the vector database, a first vector database query to cause a first vector similarity search based on a first embedded representation of the first query;
receive, from the vector database, a second ranking of the one or more documents in response to the first vector database query;
determine one or more first top documents based on the first ranking and the second ranking;
determine a first knowledge graph based on the one or more first top documents;
receive, from a first large language model (“LLM”), a first response to a prompt including the first query, the one or more first top documents, and at least a portion of the first knowledge graph;
and
output the first response to the client device.

10 more elements

Dependencies are inferred from the claim text.

Hybrid retrieval augmented generation for rich document queries using a large language model

Life of this patent

Hybrid retrieval augmented generation for rich document queries using a large language model

Life of this patent

Abstract

Claims20