US 2026/0093741 A1 · PERFORMANCE OPTIMIZATION WITH REFLECTION TOKENS IN A SELF-REFLECTIVE RETRIEVAL-AUGMENTED GENERATION FRAMEWORK

Abstract

A method includes obtaining data source tokens that identify corresponding data sources, and adding the data source tokens to a vocabulary list of a response tokenizer. An internal mapping of the response tokenizer is updated with the data source tokens, mapping each data source token to a corresponding token identifier (ID). Vector representations corresponding to the data source tokens are added to a response embedding model. The response tokenizer and the response embedding model are included in a response foundation model. An annotated dataset including multiple instances of an input utterances, retrieved data, and a data source token is created. The retrieved data is retrieved from a data source corresponding to the data source token. The response foundation model is trained with the annotated dataset to output data source tokens for a new input utterance based on a context of the new input utterance.

Claims20

Dependent claims

A method comprising:

obtaining a plurality of data source tokens, identifying a corresponding plurality of data sources;
adding the plurality of data source tokens to a vocabulary list of a response tokenizer of a response foundation model;
updating an internal mapping of the response tokenizer with the plurality of data source tokens to map each data source token of the plurality of data source tokens to a corresponding token identifier (ID);
adding a plurality of vector representations corresponding to the plurality of data source tokens to a response embedding model of the response foundation model;
creating an annotated dataset comprising a plurality of instances, each instance comprising an input utterance, a retrieved data, and a data source token, wherein the retrieved data is retrieved from a data source corresponding to the data source token;
and
training the response foundation model with the annotated dataset to output data source tokens for a new input utterance based on a context of the new input utterance.

4 more elements

A system comprising:

memory;
and
at least one computer processor configured to execute instructions for causing the system to perform operations comprising:
augmenting a prompt with at least one new data source token by a response foundation model of a response generator,
retrieving at least one new retrieved data from a new data source identified by the new data source token by a document retriever invoked by the response generator,
generating a first response segment based on the at least one new retrieved data by the response foundation model,
evaluating, by the response generator, a relevance of the first response segment,
evaluating, by the response generator, a validity of the first response segment,
evaluating, by the response generator, a utility of the first response segment, based on the relevance of the first response segment and the validity of the first response segment, and
responsive to a determination of an additional retrieval based on a context of the first response segment, augmenting the first response segment with a second new data source token by the response foundation model.

7 more elements

Dependencies are inferred from the claim text.

Performance optimization with reflection tokens in a self-reflective retrieval-augmented generation framework

Life of this patent

Performance optimization with reflection tokens in a self-reflective retrieval-augmented generation framework

Life of this patent

Abstract

Claims20