US 2026/0093704 A1Published application

Framework for generating relevant information for artificial intelligence models

AssigneePaypal, Inc.

InventorsMatan Marudi, Adva Berta Helman, Avi Simkin

Life of this patent

Estimating the term…

Loading patent…

Abstract

Methods and systems are presented for providing a retrieval-augmented generation (RAG) framework that provides information associated with a particular domain to an artificial intelligence (AI). The RAG framework includes a query generation module that generates additional queries based on an original user query. A set of documents is retrieved from a corpus based on the user query and the additional queries. The RAG framework further includes an evaluation module for evaluating a relevancy of each retrieved document with respect to the original user query. Any documents determined to be irrelevant to the original user query may be eliminated, and the remaining documents along with the original user query are used to generate a prompt for the AI model, which in turn, generates a response to the user query.

Claims20

Dependent claims

A system comprising:

a non-transitory memory;
and
one or more hardware processors coupled with the non-transitory memory and configured to execute instructions from the non-transitory memory to cause the system to:
receive, from a user device, a first query having a first scope and comprising a set of words;
generate, based on the first query, a plurality of second queries that collectively has a second scope broader than the first scope of the first query, wherein each query in the plurality of second queries is generated based on varying one or more words in the set of words of the first query;
retrieve a plurality of documents based on the plurality of second queries and the second scope, wherein the plurality of documents comprises more documents than a set of documents retrieved based on the first query and the first scope;
determine, using a machine learning model, that at least one document from the plurality of documents is not relevant to the first query based on a relevancy threshold;
provide the first query, the set of documents, and the plurality of documents, but excluding the at least one document, to an artificial intelligence (AI) model, wherein the AI model is configured to generate a response to the first query based on the set of documents and the plurality of documents excluding the at least one document;
and
transmit the response to the user device.

7 more elements

A method comprising:

receiving, by a computer system, an utterance from a user device via a chat interface;
generating, by the computer system and based on the utterance, a first query having a first scope and comprising a set of words;
generating, by the computer system, a plurality of different queries based on modifying one or more words in the set of words of the first query, wherein one or more of the plurality of queries have a second scope that is broader than the first scope;
querying, using the plurality of different queries, a database for a plurality of documents, wherein the plurality of documents comprises more documents than a set of documents retrieved based on the first query and the first scope;
determining, using a machine learning model, that at least one document from the plurality of documents is not relevant to the first query based on a set of relevancy criteria;
generating, by the computer system, a prompt for an artificial intelligence (AI) model based on the first query, the set of documents, and a subset of the plurality of documents, wherein the subset of the plurality of documents excludes the at least one document;
providing the prompt to the AI model, wherein the AI model is configured to generate a response to the utterance based on content included in the set of documents and the subset of the plurality of documents;
and
transmitting, by the computer system, the response to the user device via the chat interface.

6 more elements

A non-transitory machine-readable medium having stored thereon machine-readable instructions executable to cause a machine to perform operations comprising:

receiving an utterance from a user device;
generating, based on the utterance, a first query having a first scope and comprising a set of words;
generating a plurality of different queries based on modifying one or more words in in the set of words of the first query, wherein a second scope associated with the plurality of different queries is broader than the first scope;
querying a database for a plurality of documents based on the plurality of different queries and the second scope, wherein the plurality of documents comprises more documents than a set of documents retrieved based on the first query and the first scope;
determining, using a machine learning model, that at least one document from the plurality of documents is not relevant to the first query based on a relevancy threshold;
generating a prompt for an artificial intelligence (AI) model based on the first query, the set of documents, and a subset of the plurality of documents, wherein the subset of the plurality of documents excludes the at least one document;
providing the prompt to the AI model, wherein the AI model is configured to generate a response to the utterance based on content included in the set of documents and the subset of the plurality of documents;
and
transmitting the response to the user device.

6 more elements

Dependencies are inferred from the claim text.