System and method for determination of label values in unstructured documents

A system (500) for determining label value for labels in unstructured documents, comprising:

• one or more memories (515);
• and
• one or more hardware processors (502), the one or more memories (515) coupled to the one or more hardware processors (502), wherein the one or more hardware processors (502) are configured to execute programmed instructions stored in the one or more memories (515), to:
• define an extraction profile comprising a set of labels for which label values are to be extracted from the unstructured document;
• identify a plurality of sections in one or more page images of the unstructured document, each section of the plurality of sections identified based on one or more image processing techniques;
• generate a plurality of bounding boxes in the one or more page images, each of the plurality of bounding boxes enclosing a section of the plurality of sections;
• obtain a label value for each label from amongst the plurality of labels stored in the extraction profile, wherein obtaining the label value for each label comprises:
• extract the plurality of labels, wherein extracting a label comprises performing for each bounding box of the plurality of bounding boxes:
• extract text comprised in the bounding box, features of the bounding box and an Optical character recognition (OCR) confidence score (COCR) associated with the text based on a confidence score associated with each word of the text using an OCR technique;
• determine whether a label text for a label from amongst the plurality of labels is present in the bounding box, the label text for the label comprising one of a label name and one or more synonyms for the label name;
• on determination of absence of the label in the bounding box, apply an OCR error correction model and a partial matching model, wherein the OCR error correction model utilizes a minimum distance technique to identify inaccuracy in the text identified through the OCR technique, and the partial matching model computes a level of matching between the text identified using the OCR error correction model and the label from amongst the plurality of labels;
• and
• extract the label from the bounding box on determination of the level of matching between the text identified using the OCR error correction model and the label from amongst the plurality of labels being more than or equal to a predefined threshold;
• identify, from amongst the plurality of bounding boxes, a bounding box comprising label value corresponding to the label, the bounding box being one of the bounding boxes comprising the label text and a value matching data type criteria for the label in the bounding box and a neighboring bounding box containing value matching data type criteria in vicinity of the bounding box comprising the label text using a nearest proximity neighbor criteria;
• predict a bounding box comprising the label value associated with the label text using a deep learning model, the deep learning model trained with location information and data type criteria associated with the label values of the labels;
• and
• obtain an aggregate confidence score for the text in the bounding box indicative of the text being a label value for the label in the bounding box, the aggregate confidence score obtained as a weighted sum of a confidence score (CPOS) of identification of position of the bounding box comprising label value in comparison with the position of label value obtained from deep learning model, a confidence score (CPROXIMITY) of the extracted value of the label in a neighboring bounding box, a confidence score (CSIZE) associated with size of the bounding box, and the COCR associated with the OCR.

One or more non-transitory machine readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors cause:

• defining, via the one or more hardware processors, an extraction profile comprising a set of labels for which label values are to be extracted from the unstructured document;
• identifying, via the one or more hardware processors, a plurality of sections in one or more page images of the unstructured document, each section of the plurality of sections identified based on one or more image processing techniques;
• generating, via the one or more hardware processors, a plurality of bounding boxes in the one or more page images, each of the plurality of bounding boxes enclosing a section of the plurality of sections;
• obtaining, via the one or more hardware processors, a label value for each label from amongst the plurality of labels stored in the extraction profile, wherein obtaining the label value for each label comprises:
• extracting the plurality of labels, via the one or more hardware processors, wherein extracting a label comprises performing for each bounding box of the plurality of bounding boxes:
• extracting text comprised in the bounding box, features of the bounding box, and an Optical character recognition (OCR) confidence score (COCR) associated with the text based on a confidence score associated with each word of the text using an OCR technique;
• determining whether a label text for a label from amongst the plurality of labels is present in the bounding box, the label text for the label comprising one of a label name and one or more synonyms for the label name;
• on determination of absence of the label in the bounding box, applying an OCR error correction model and a partial matching model, wherein the OCR error correction model utilizes a minimum distance technique to identify inaccuracy in the text identified through the OCR technique, and the partial matching model computes a level of matching between the text identified using the OCR error correction model and the label from amongst the plurality of labels;
• and
• extracting the label from the bounding box on determination of the level of matching between the text identified using the OCR error correction model and the label from amongst the plurality of labels being more than or equal to a predefined threshold;
• identifying, from amongst the plurality of bounding boxes, a bounding box comprising label value corresponding to the label, the bounding box being one of the bounding boxes comprising the label text and a value matching data type criteria for the label in the bounding box and a neighboring bounding box containing value matching data type criteria in vicinity of the bounding box comprising the label text using a nearest proximity neighbor criteria;
• predicting, via the one or more hardware processors, a bounding box comprising the label value associated with the label text using a deep learning model, the deep learning model trained with location information and data type criteria associated with the label values of the labels;
• and
• obtaining, via the one or more hardware processors, an aggregate confidence score for the text in the bounding box indicative of the text being a label value for the label in the bounding box, the aggregate confidence score obtained as a weighted sum of a confidence score (CPOS) of identification of position of the bounding box comprising label value in comparison with the position of label value obtained from deep learning model, a confidence score (CPROXIMITY) of the extracted value of the label in a neighboring bounding box, a confidence score (CSIZE) associated with size of the bounding box, and the COCR associated with the OCR.