# US 2021/0334530 A1 · Data extraction from short business documents > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0334530 A1 - Document ID: US20210334530A1 - Status: Published application - Kind: A1 - Application number: 17/252,678 - Filed: 2019-06-21 - Published: 2021-10-28 - Earliest claimed: 2018-06-21 - Assignees: Element Ai Inc. - Applicants: Element Ai Inc. - Inventors: Elena BUSILA; Jerome PASQUERO; Tim BEIKO; Evelin FONSECA CRUZ; Minh-Kim DAO; Majid LAALI; Patrick LAZARUS - Web page: https://patentgrep.com/patent/US20210334530A1 ## CPC - G06V 30/414: Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text - G06F 16/906: Clustering; Classification - G06N 20/00: Machine learning - G06T 5/73: Deblurring; Sharpening - G06T 5/80: Geometric correction - G06V 30/133: {Evaluation of quality of the acquired characters} - G06V 30/413: Classification of content, e.g. text, photographs or tables - G06V 30/416: Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors - G06T 2207/20132: Image cropping - G06V 30/10: Character recognition - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Systems and methods for document analysis. An image containing at least one document is received at a pre-processing stage and the image is analyzed for image quality. If the image quality is insufficient for further processing, this is adjusted until the image is suitable for further processing. After the image quality adjustment, the image is then passed to an initial processing stage. At the initial processing stage, the boundaries of one or more documents within the image are determined. In addition, the orientation of the image may be adjusted and the type of document(s) within the image is determined. From the initial processing stage, the adjusted image is then passed to a data extraction stage. At this stage, clusters of data within the document are determined and bounding boxes are placed around the clusters. Data regarding each of the clusters of data is then gathered. ## Claims ### Claim 1 A system for processing an image containing at least one document, the system comprising: a pre-processing stage for ensuring that an image quality of said image is suitable for processing; an initial processing stage for determining at least one boundary of said at least one document, said initial processing stage receiving an output of said pre-processing stage; a data extraction stage for determining clusters of data in said at least one document and for determining data regarding said clusters of data, said data extraction stage receiving an output of said initial processing stage; wherein said system produces an output suitable for further processing to determine contents of said at least one document. ### Claim 2 (depends on 1) The system according to claim 1, wherein said image contains multiple documents and said initial processing stage determines boundaries for said multiple documents. ### Claim 3 (depends on 1) The system according to claim 1, wherein said pre-processing stage performs one or more actions on said image, said actions being one or more of: removal of artefacts from said image; adjusting a contrast of said image; cropping said image; adjusting a geometry of said image; adjusting said image to compensate for projection effects; adjusting a sharpness of said image; and adjusting a color of said image. ### Claim 4 (depends on 1) The system according to claim 1, wherein said initial processing stage of said system performs one or more actions on said image, said actions being one or more of: performing a rotation of said image; performing a partial rotation of said image; performing a geometric translation of said image; performing at least one image adjustment to compensate for projection effects; determining a top portion of said at least one document in said image; determining a bottom portion of said at least one document in said image; determining a delineation of a page in said at least one document in said image using a detection of either headers or footers in said page; determining a type of said at least one document; and performing an OCR process on at least one portion of said at least one document. ### Claim 5 (depends on 1) The system according to claim 1, wherein said data extraction stage of said system performs one or more actions on said image, said actions being one or more of: determining various types of clusters of data in said at least one document in said image; placing bounding boxes around said clusters of data; determining a position of each cluster of data within said at least one document; concatenating adjacent clusters of data when necessary to form a single cluster of data; for at least one cluster of data containing text data, determining a font type used in said text data; for at least one cluster of data containing text data, determining a font size used in said text data; performing an OCR process on one or more clusters of data containing text data to determine a content of said one or more clusters of data; for at least one cluster of data containing image data, determining shapes present in said image data; for at least one cluster of data containing image data, determining lines present in said image data; for at least one cluster of data containing image data, determining if said image data contains a logo; for at least one cluster of data containing image data, determining if said image data contains a bar code; determining if a portion of said at least one document indicates a check box being filled in; determining if a portion of said at least one document indicates a radio button being activated; for at least one cluster of data containing image data, determining if said image data contains alphanumeric characters; and for each cluster of data, determining a size in pixels of said cluster. ### Claim 6 (canceled) ### Claim 7 (canceled) ### Claim 8 (depends on 1) The system according to claim 1, wherein said initial processing stage determines boundaries for multiple documents in said image. ### Claim 9 (canceled) ### Claim 10 (depends on 1) The system according to claim 1, further comprising a neural network to receive an output of said data extraction stage, said neural network being for determining a type of data present in each cluster of data in said document. ### Claim 11 A method for determining at least one type of data present in a business-related document, the method comprising: a) receiving an image of said business-related document; b) defining clusters of data on said document from said image; c) determining a type of data present in a specific cluster based on at least one of: said specific cluster's position within said document and a content of said specific cluster. ### Claim 12 (depends on 11) The method according to claim 11, further comprising a step of determining said type of data present in said specific cluster based on said specific cluster's position relative to other clusters in said document. ### Claim 13 (depends on 11) The method according to claim 11, wherein step c) further comprises determining said type of data present in said specific cluster based on a size of said specific cluster. ### Claim 14 (depends on 11) The method according to claim 11, wherein step c) further comprises determining said type of data present in said specific cluster based on a content of said specific cluster. ### Claim 15 (canceled) ### Claim 16 (canceled) ### Claim 17 A system for determining at least one type of data present in a business-related document, the system comprising: an image reception module for receiving an image of said document; a cluster determination module for determining clusters of data in said document using said image; a data determination module for determining a type of data present in at least one of specific cluster based on a position of said at least one specific cluster in said document. ### Claim 18 (depends on 17) The system according to claim 17, wherein said system further comprises a position determination module for determining positions of said clusters of data in said document. ### Claim 19 (canceled) ### Claim 20 (depends on 17) The system according to claim 17, wherein said at least one type of data comprises address data, date data, and total amount data. ### Claim 21 (depends on 17) The system according to claim 17, wherein at least one of said modules comprises a neural network. ### Claim 22 (depends on 17) The system according to claim 17, wherein at least one of said modules is a rule based module. ### Claim 23 (depends on 17) The system according to claim 17, wherein, in addition to said position of said specific cluster in said document, said data determination module determines said type of data based on said specific cluster's position relative to other clusters in said document. ### Claim 24 (depends on 17) The system according to claim 17, wherein, in addition to said position of said specific cluster in said document, said data determination module determines said type of data based on a content of said specific cluster. ### Claim 25 (depends on 17) The system according to claim 17, wherein, in addition to said position of said specific cluster in said document, said data determination module determines said type of data based on a size of said specific cluster. ### Claim 26 (canceled) ### Claim 27 (depends on 17) The system according to claim 17, wherein at least one of said modules is implemented using a combination of machine learning and rules based methods.