Response generation using multimodal retrieval augmented generation
AssigneeCisco Technology, Inc.
InventorsRamin Pishehvar, Mahesh Viswanathan, Ali Payani, Nima Nafisi
Life of this patent
Estimating the term…
Abstract
In one implementation, a device receives an input query from a requester. The device uses a multimodal large language model to generate an output based on the input query. The device identifies one or more files by performing a search of an information retrieval index based on the input query. The device provides, to the requester, a response to the input query that is based on the output of the multimodal large language model and on the one or more files.
Claims20
A method, comprising:
- receiving, at a device, an input query from a requester;
- using, by the device, a multimodal large language model to generate an output based on the input query;
- identifying, by the device, one or more files by performing a search of an information retrieval index based on the input query;
- generating, by the device, a multimodal response by inserting one or more references to the one or more files into the output of the multimodal large language model, wherein the one or more references comprise one or more selectable links to the one or more files;
- and
- causing, by the device, a display to present the multimodal response to the requester, wherein the display renders the one or more selectable links to enable the requester to access the one or more files.
An apparatus, comprising:
- one or more network interfaces;
- a processor coupled to the one or more network interfaces and configured to execute one or more processes;
- and
- a memory configured to store a process that is executable by the processor, the process when executed configured to:
- receive an input query from a requester;
- use a multimodal large language model to generate an output based on the input query;
- identify one or more files by performing a search of an information retrieval index based on the input query;
- generate a multimodal response by inserting one or more references to the one or more files into the output of the multimodal large language model, wherein the one or more references comprise one or more selectable links to the one or more files;
- and
- cause a display to present the multimodal response to the requester, wherein the display renders the one or more selectable links to enable the requester to access the one or more files.
A tangible, non-transitory, computer-readable medium storing program instructions that cause a device to execute a process comprising:
- receiving, at the device, an input query from a requester;
- using, by the device, a multimodal large language model to generate an output based on the input query;
- identifying, by the device, one or more files by performing a search of an information retrieval index based on the input query;
- generating, by the device, a multimodal response by inserting one or more references to the one or more files into the output of the multimodal large language model, wherein the one or more references comprise one or more selectable links to one or more specific segments within the one or more files;
- and
- causing, by the device, a display to present the multimodal response to the requester, wherein the display renders the one or more selectable links to enable the requester to access the specific segments of the one or more files.
Dependencies are inferred from the claim text.