US 2026/0187148 A1Published application

Method and system for implementing retrieval augmented generation with improved context identification from semi-structured documents

AssigneeLTI Mindtree Ltd

InventorsBrijesh Prabhakar, ChandiPrasad Ojha, Sibin Kunnumpurath James

Life of this patent

Estimating the term…

Loading patent…

Abstract

Method and system for implementing RAG with improved context identification from semi-structured documents is disclosed. A target document is preprocessed by segmenting into individual pages. Text content from each individual page is extracted and processed through a domain-specific token filter to identify terms matching a stored domain-specific dictionary, and generate filtered page content comprising the matching terms. A page vector representation of the filtered page is computed, and vector similarity is performed between each page vector representation and stored vectors in a vector database that are associated with predefined entity types via entity labels that are extracted from matched templates. The entity labels are mapped to corresponding document pages based on similarity scores. A user query is analyzed using a Large Language Model (LLM) to identify relevant entity labels and pages with the relevant entity labels to generate a response.

Claims26

Dependent claims

A computer-implemented method for identifying relevant content from semi-structured documents, the method comprising:

receiving, by a processor, a target document and a user query to initiate text extraction operations from the target document;
preprocessing, by the processor, the target document by:
segmenting the target document into individual pages;
extracting text content from each individual page by:
processing the extracted text content through a domain-specific token filter to identify terms matching a stored domain-specific dictionary, and generate filtered page content comprising the matching terms;
computing, using an embedding model, a page vector representation of the filtered page content;
performing vector similarity computation between each page vector representation and stored template vectors in a vector database, wherein each stored template vector is associated with predefined entity types via entity labels;
extracting the entity labels from matched templates and mapping the entity labels to corresponding document pages based on similarity scores;
processing, by the processors the user query by:
analyzing, using a Large Language Model (LLM), the user query to identify relevant entity labels from the mapped document pages;
identifying one or more pages containing the relevant entity labels;
and
generating, by the processor, a response to the user query using the LLM by sending the user query and the identified pages.

10 more elements

A system for identifying relevant text from semi-structured documents, the system comprising:

a processor;
a memory storing instructions that, when executed by the processor, cause the processor to:
receive a target document and a user query to initiate text extraction operations from the target document;
preprocess the target document by:
segmenting the target document into individual pages;
extracting text content from each individual page by:
process the extracted text content through a domain-specific token filter to identify terms matching a stored domain-specific dictionary, and generate filtered page content comprising the matching terms;
computing, using an embedding model, a page vector representation of the filtered page content;
performing vector similarity computation between each page vector representation and stored template vectors in a vector database, wherein each stored template vector is associated with predefined entity types via entity labels;
extracting the entity labels from matched template and map the entity labels to corresponding document pages based on similarity scores;
process the user query by:
analyzing, using a Large Language Model (LLM), the user query to identify relevant entity labels from the mapped document pages;
identifying one or more pages containing the relevant entity labels;
and
generate a response to the user query by calling LLM with the user query and identified pages.

12 more elements

Dependencies are inferred from the claim text.