Effective Multi-Modal Retrieval-Augmented Generation (RAG) Agent With Twin-Database And Comprehensive Multi-Format Data Ingestion
AssigneeOracle International Corporation
InventorsLiyu Gong, Yuying Wang
Life of this patent
Estimating the term…
Loading patent…
AssigneeOracle International Corporation
InventorsLiyu Gong, Yuying Wang
Estimating the term…
Techniques for ingesting and using content items by a Retrieval-Augmented Generation (RAG) agent are disclosed. A RAG agent accesses content items that include textual data and/or non-textual image data (e.g., a table, a chart, a document, or a picture). When the RAG agent detects that content items include non-textual image data, the RAG agent invokes a large multimodal model (LMM) that is configured to classify the non-textual image data into a variety of classifications. The RAG agent also classifies the non-textual image data. Using this classification as selection criteria, the RAG agent selects an LMM that corresponds to the classification from a set of available LMMs. The RAG agent ensures that the selected LMM is configured to generate text from non-textual image data that corresponds to the classification. The generated text and extracted image data are both used by the RAG agent to respond to queries.
One or more non-transitory computer readable media comprising instructions which, when executed by one or more hardware processors, cause performance of operations comprising:
A method comprising:
A system comprising:
Dependencies are inferred from the claim text.