US 2026/0065706 A1 · SYSTEM AND METHOD FOR AUTOMATED EXTRACTION OF CONTEXTUAL INFORMATION FROM DATA USING LARGE MULTIMODAL MODEL

Abstract

A system and method for automated extraction of contextual information from a set of data using an LMM is provided. The method begins by splitting the set of data into sub-data based on logical boundaries and converting each sub-data into an image. These images are then processed using a computer vision model and optical image recognition to extract metadata and positional coordinates. The images, extracted metadata, and positional coordinates are integrated into a custom prompt for the LMM. This prompt is processed to obtain relevant data points, which are subsequently validated for accuracy using a trained validation model that considers content density versus output records, regex pattern-based record matching scores, and template-based records approximation scores. An LLM is then used to normalize headers in the relevant data points. Finally, the method automatically extracts contextual information by generating responses to user queries on the normalized relevant data points using the LLM.

Claims10

Dependent claims

A method for automated extraction of contextual information from a set of data using a large multimodal model (LMM) (108), the method comprising:

splitting the set of data into one or more sub-data based on logical boundaries and converting each sub-data into an image;
processing the image using a computer vision model and an optical image recognition method to obtain extracted metadata and positional coordinates of the extracted metadata;
integrating the image with the extracted metadata and the positional coordinates of the extracted metadata into a custom prompt for the LMM (108);
processing the custom prompt for the LMM (108) to obtain relevant data points;
validating the relevant data points for accuracy using a validation model that is trained on features comprising content density versus output records, a regex pattern-based record matching score and a template-based records approximation score;
processing a prompt with a large language model (LLM) (110) to normalize headers in the relevant data points to obtain normalized relevant data points;
and
generating response to queries on the normalized relevant data points obtained from a user using the LLM (110) for performing automated extraction of contextual information from the set of data using the LMM (108).

5 more elements

A system for automated extraction of contextual information from a set of data using a large multimodal model (LMM) (108), the system comprising:

an automated extraction server (104) comprising a processor and a memory being configured to perform:
splitting the set of data into one or more sub-data based on logical boundaries and converting each sub-data into an image;
processing the image using a computer vision model and an optical image recognition method to obtain extracted metadata and positional coordinates of the extracted metadata;
validating the extracted metadata for accuracy using a validation model that is trained on features comprising content density versus output records, a regex pattern-based record matching score and a template-based records approximation score;
integrating the image with the extracted metadata and the positional coordinates of the extracted metadata into a custom prompt for the LMM (108);
processing the custom prompt for a large language model (LLM) (110) to obtain relevant data points;
processing a prompt with the LLM (110) to normalize headers in the relevant data points to obtain normalized relevant data points;
and
generating response to queries on the normalized relevant data points obtained from a user using the LLM (110) for performing automated extraction of contextual information from the set of data using the LMM (108).

6 more elements

Dependencies are inferred from the claim text.

System and method for automated extraction of contextual information from data using large multimodal model

Life of this patent

System and method for automated extraction of contextual information from data using large multimodal model

Life of this patent

Abstract

Claims10