US 2026/0310702 A1Published application

Native retrieval augmented generation model(s)

AssigneeGoogle Inc.

InventorsZhe Dong, Fedor Moiseev, Henrique Schechter Vera, Sahil Dua, Paul Suganthan G C

Life of this patent

Estimating the term…

Loading patent…

Abstract

Various implementations include a method for processing a natural language user query using a native retrieval augmented generation (NatRAG) model. The method includes receiving a natural language (NL) user query and processing it using a NatRAG model comprising a generative portion with decoder layers and a retrieval portion. Processing involves generating NatRAG input, processing it with the generative portion to produce output tokens, and determining if a “need to retrieve” (N-t-R) token is present. If so, the retrieval portion processes decoder layers to generate retrieval tokens, which are appended to the input to create updated NatRAG input. This updated input is then processed by the NatRAG model to generate output, enabling the device to perform actions based on the generated output.

Claims20

Dependent claims

A method implemented by one or more processors, the method comprising:

receiving a natural language (NL) user query at a computing device;
processing the NL user query using a native retrieval augmented generation (NatRAG) model to generate NatRAG output responsive to the NL user query, wherein the NatRAG model includes a generative portion and a retrieval portion, wherein processing the NL user query using the NatRAG model to generate NatRAG output comprises:
generating NatRAG input based on at least the NL user query;
processing the NatRAG input using the generative portion of the NatRAG model to generate one or more output tokens, in a sequence of output tokens, wherein the generative portion of the NatRAG model includes at least a set of decoder layers;
determining whether the one or more output tokens includes a need to retrieve (N-t-R) token, wherein the N-t-R token indicates the generative portion of the NatRAG model requires additional information;
processing one or more portions of the set of decoder layers using the retrieval portion of the NatRAG model to generate one or more retrieval tokens in a sequence of retrieval tokens;
appending the one or more retrieval tokens to the NatRAG input to generate updated NatRAG input;
processing the updated NatRAG input using the NatRAG model to generate NatRAG output responsive to the NL user query;
and
causing the computing device to perform one or more actions based on the NatRAG output.

7 more elements

A system comprising:

memory storing instructions;
one or more processors configured to execute the instructions to:
receive a natural language (NL) user query at a computing device;
process the NL user query using a native retrieval augmented generation (NatRAG) model to generate NatRAG output responsive to the NL user query, wherein the NatRAG model includes a generative portion and a retrieval portion, wherein processing the NL user query using the NatRAG model to generate NatRAG output comprises:
generate NatRAG input based on at least the NL user query;
process the NatRAG input using the generative portion of the NatRAG model to generate one or more output tokens, in a sequence of output tokens, wherein the generative portion of the NatRAG model includes at least a set of decoder layers;
determine whether the one or more output tokens includes a need to retrieve (N-t-R) token, wherein the N-t-R token indicates the generative portion of the NatRAG model requires additional information;
process one or more portions of the set of decoder layers using the retrieval portion of the NatRAG model to generate one or more retrieval tokens in a sequence of retrieval tokens;
append the one or more retrieval tokens to the NatRAG input to generate updated NatRAG input;
process the updated NatRAG input using the NatRAG model to generate NatRAG output responsive to the NL user query;
and
causing the computing device to perform one or more actions based on the NatRAG output.

9 more elements

A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors of a system, cause the system to:

receive a natural language (NL) user query at a computing device;
process the NL user query using a native retrieval augmented generation (NatRAG) model to generate NatRAG output responsive to the NL user query, wherein the NatRAG model includes a generative portion and a retrieval portion, wherein processing the NL user query using the NatRAG model to generate NatRAG output comprises:
generate NatRAG input based on at least the NL user query;
process the NatRAG input using the generative portion of the NatRAG model to generate one or more output tokens, in a sequence of output tokens, wherein the generative portion of the NatRAG model includes at least a set of decoder layers;
determine whether the one or more output tokens includes a need to retrieve (N-t-R) token, wherein the N-t-R token indicates the generative portion of the NatRAG model requires additional information;
process one or more portions of the set of decoder layers using the retrieval portion of the NatRAG model to generate one or more retrieval tokens in a sequence of retrieval tokens;
append the one or more retrieval tokens to the NatRAG input to generate updated NatRAG input;
process the updated NatRAG input using the NatRAG model to generate NatRAG output responsive to the NL user query;
and
causing the computing device to perform one or more actions based on the NatRAG output.

7 more elements

Dependencies are inferred from the claim text.