Efficient rag model for medical applications
AssigneeDatum Point Labs Inc.
InventorsDAVID LEE, HANSEOK KO, BONHWA KU, NOJIN PARK
Life of this patent
Estimating the term…
Abstract
Embodiments described herein provide systems and methods for retrieval augmented generation. Embodiments herein include a pipeline for database construction from unlabeled data. Embodiments also include smart chunking techniques for more efficient retrieval. Embodiments also include quantization of a sentence embedding model used in the retrieval process, resulting in a faster more lightweight overall system. Use of a lightweight LLM allows for local LLM inference, increasing data privacy.
Claims20
A method of retrieval augmented generation, comprising:
- receiving, via a data interface, a query;
- retrieving a plurality of text chunks from a database based on a comparison of contents of the database with an embedding of the query;
- inputting a prompt to a neural network based language model, the prompt including the plurality of text chunks and the query;
- and
- outputting, via the neural network based language model, a text output based on the prompt.
A system for retrieval augmented generation, comprising:
- a memory storing processor executable instructions;
- and
- one or more processors that read and execute the processor executable instructions from the memory to perform operations comprising:
- receiving, via a data interface, a query;
- retrieving a plurality of text chunks from a database based on a comparison of contents of the database with an embedding of the query;
- inputting a prompt to a neural network based language model, the prompt including the plurality of text chunks and the query;
- and
- outputting, via the neural network based language model, a text output based on the prompt.
A non-transitory machine-readable medium comprising a plurality of machine-executable instructions which, when executed by one or more processors, are adapted to cause the one or more processors to perform operations comprising:
- receiving, via a data interface, a query;
- retrieving a plurality of text chunks from a database based on a comparison of contents of the database with an embedding of the query;
- inputting a prompt to a neural network based language model, the prompt including the plurality of text chunks and the query;
- and
- outputting, via the neural network based language model, a text output based on the prompt.
Dependencies are inferred from the claim text.