Method and system for extracting contents from documents into predefined format

A system comprising:

• a memory storing instructions;
• one or more Input/Output (I/O) interfaces;
• and
• one or more hardware processors coupled to the memory via the one or more communication interfaces, wherein the one or more hardware processors are configured by the instructions to:
• receive a text document comprising one or more pages of textual information;
• convert the one or more pages into one or more images;
• divide each of the one or more images into a plurality of bounding boxes;
• iteratively process each image of the one or more images until a covered height is less than total height of the plurality of bounding boxes in each of the one or more images, by:
• cropping each of the one or more images along a first bounding box among the plurality of bounding boxes to obtain a corresponding cropped image;
• and
• processing the cropped image corresponding to each of the one or more images, via an Optical Character Recognition (OCR) reader,
• wherein if output of the OCR reader is an empty string, then updating covered height with height of the first bounding box,
• wherein if output of the OCR reader is a non-empty string, then checking completeness of the first bounding box, checking completeness of an equation within the first bounding box, and extracting text within the first bounding box using a multi-modal large language model, wherein the cropping of each of the one or more images and the processing of corresponding cropped images is performed for remaining bounding boxes from among the plurality of bounding boxes to extract text from the plurality of bounding boxes, and wherein the covered height is updated with height of the remaining bounding boxes being processed;
• and
• consolidate text extracted from the plurality of bounding boxes in the one or more images into a predefined format.

One or more non-transitory machine-readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors cause:

• receiving a text document comprising one or more pages of textual information;
• converting the one or more pages into one or more images;
• dividing each of the one or more images into a plurality of bounding boxes;
• iteratively processing each image of the one or more images until a covered height is less than total height of the plurality of bounding boxes in each of the one or more images, by:
• cropping each of the one or more images along a first bounding box among the plurality of bounding boxes to obtain a corresponding cropped image;
• and
• processing the cropped image corresponding to each of the one or more images, via an Optical Character Recognition (OCR) reader,
• wherein if output of the OCR reader is an empty string, then updating covered height with height of the first bounding box,
• wherein if output of the OCR reader is a non-empty string, then checking completeness of the first bounding box, checking completeness of an equation within the first bounding box, and extracting text within the first bounding box using a multi-modal large language model, wherein the cropping of each of the one or more images and the processing of corresponding cropped images is performed for remaining bounding boxes from among the plurality of bounding boxes to extract text from the plurality of bounding boxes, and wherein the covered height is updated with height of the remaining bounding boxes being processed;
• and
• consolidating text extracted from the plurality of bounding boxes in the one or more images into a predefined format.