System and method for extracting structured information from image documents

A non-transitory computer-readable medium storing software comprising instructions that are executable by one or more device processors to extract structured information from image documents by:

• obtaining an input image document including text;
• analyzing the input image document to determine a skeletal layout of information included in the input image document by removing the text from the input image document to determine the skeletal layout that includes a fingerprint of the input image document, wherein the fingerprint of the input image document includes a multi-dimensional feature vector representing boxes in the input image document;
• performing matching of the determined skeletal layout against a plurality of document templates to determine a measure of similarity between the determined skeletal layout and each of the document templates by:
• comparing the determined fingerprint of the input image document to a plurality of fingerprints including multi-dimensional feature vectors representing the plurality of document templates;
• and
• determining a measure of similarity between the fingerprint of the input image and the plurality of fingerprints representing the plurality of document templates by determining Euclidean distance between the multi-dimensional feature vectors of each;
• selecting one of the document templates as a matched template, based on a determination that the determined measure of similarity between the determined skeletal layout and the selected document template exceeds a predetermined threshold similarity value;
• cropping out a plurality of box areas from the input image document, based on a result of the selecting of a matched template;
• obtaining results of optical character recognition (OCR) on the cropped plurality of box areas, the results including recognized text recognized from image data located within the plurality of box areas in the input image document;
• automatically processing the obtained recognized text using directed search based on a list of historical OCR errors;
• correcting the errors found in the obtained recognized text by using the results of the directed search;
• classifying at least a portion of the recognized text into one or more classification categories using statistical language modeling;
• providing the classified recognized text to a processor for processing the input image document in accordance with the one or more categories;
• and
• generating a report based on a result of the processing of the input image document.

A system for extracting structured information from image documents, the system comprising:

• a device processor;
• and
• a non-transitory computer readable medium storing instructions that are executable by the device processor to:
• obtain an input image document including text;
• analyze the input image document to determine a skeletal layout of information included in the input image document by removing the text from the input image document to determine the skeletal layout that includes a fingerprint of the input image document, wherein the fingerprint of the input image document includes a multi-dimensional feature vector representing boxes in the input image document;
• perform matching of the determined skeletal layout against a plurality of document templates to determine a measure of similarity between the determined skeletal layout and each of the document templates by:
• compare the determined fingerprint of the input image document to a plurality of fingerprints including multi-dimensional feature vectors representing the plurality of document templates;
• and
• determine a measure of similarity between the fingerprint of the input image and the plurality of fingerprints representing the plurality of document templates by determining Euclidean distance between the multi-dimensional feature vectors of each;
• select one of the document templates as a matched template, based on a determination that the determined measure of similarity between the determined skeletal layout and the selected document template exceeds a predetermined threshold similarity value;
• crop out a plurality of box areas from the input image document, based on a result of the selecting of a matched template;
• obtain results of optical character recognition (OCR) on the cropped plurality of box areas, the results including recognized text recognized from image data located within the plurality of box areas in the input image document;
• automatically process the obtained recognized text using directed search based on a list of historical OCR errors;
• correct the errors found in the obtained recognized text by using the results of the directed search;
• classify at least a portion of the recognized text into one or more classification categories using statistical language modeling;
• provide the classified recognized text to a processor for processing the input image document in accordance with the one or more categories;
• and
• generate a report based on a result of the processing of the input image document.