# US 2021/0240975 A1 · Document spatial layout feature extraction to simplify template classification > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0240975 A1 - Document ID: US20210240975A1 - Status: Published application - Kind: A1 - Application number: 16/779,462 - Filed: 2020-01-31 - Published: 2021-08-05 - Earliest claimed: 2020-01-31 - Assignees: Automation Anywhere, Inc. - Applicants: Automation Anywhere, Inc. - Inventors: Michael Sundell; Vibhas Gejji - Web page: https://patentgrep.com/patent/US20210240975A1 ## CPC - G06V 30/414: Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text - G06F 16/93: Document management systems - G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)} - G06F 18/24: Classification techniques - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/08: Learning methods - G06N 3/09: Supervised learning - G06V 30/18057: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]} - G06V 30/19173: {Classification techniques} - G06V 30/412: Layout analysis of documents structured with printed lines or input boxes, e.g. business forms or tables - G06V 30/413: Classification of content, e.g. text, photographs or tables - G06V 30/10: Character recognition - G06V 2201/09: Recognition of logos - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract Image encoded documents are identified by recognizing known objects in each document with an object recognizer. The objects in each page are filtered to remove lower order objects. Known features in the objects are recognized by sequentially organizing each object in each filtered page into a one-dimensional array, where each object is positioned in a corresponding one-dimensional array as a function of location in the corresponding filtered page. The one-dimensional array is then compared to known arrays to classify the image document corresponding to the one-dimensional array. ## Claims ### Claim 1 A document processing system comprising: data storage for storing a plurality of documents in image format, the documents comprising one or more sets of documents, each set of documents comprising documents of having similar formatting; and a processor programmed with instructions that when executed implement: an object recognizer that processes documents in a set of documents and generates for each document in the set of documents, a set of page objects, wherein the page objects are organized in a two-dimensional arrangement in a corresponding document; a feature extractor responsive to a set of page objects for extracting known features from each set of page objects by sequentially organizing each object in each set of page objects into a one-dimensional array, each object positioned in a corresponding one-dimensional array as a function of location in the corresponding set of page objects; a template identifier for processing each one-dimensional array by comparing each one-dimensional array to a plurality of known one-dimensional arrays, each of the known one-dimensional arrays corresponding to an image encoded document having a known formatting; and classifying the document in image format corresponding to the one-dimensional array into a classification having a known formatting upon a match within a predetermined match threshold. ### Claim 2 (depends on 1) The document processing system of claim 1 wherein the feature extractor sequentially organizes each object in each filtered page into a one-dimensional array as a function of location of each object from top-to-bottom and then left-to-right in the corresponding filtered page. ### Claim 3 (depends on 2) The document processing system of claim 2 wherein the feature extractor further converts location of each object into a first vector measured from a top-right corner from the corresponding filtered page to center of the object and into a second vector measured from a bottom-right corner from the corresponding filtered page to center of the object and selects the first or the second vector as a function of which of the first or the second vector has a larger magnitude. ### Claim 4 (depends on 1) The document processing system of claim 1 wherein the object recognizer comprises a deep neural network trained to identify objects in at least one of the sets of documents and to generate for each document processed by the deep neural network, a set of page objects. ### Claim 5 (depends on 1) The document processing system of claim 1 wherein the object recognizer comprises an optical character recognizer that generates for each document processed by the optical character recognizer, a set of page objects. ### Claim 6 (depends on 1) The document processing system of claim 1 further comprising a higher order object filter for processing each set of page objects organized in a two dimensional arrangement to generate for each set of page objects a filtered page by removing lower order objects from the set of page objects and wherein the filtered page is provided to the feature extractor as the set of page objects. ### Claim 7 (depends on 2) The document processing system of claim 2 wherein the feature extractor further converts location of each object into a first vector measured from a top-left corner from the corresponding filtered page to center of the object and into a second vector measured from a bottom-left corner from the corresponding filtered page to center of the object and selects the first or the second vector as a function of which of the first or the second vector has a larger magnitude. ### Claim 8 (depends on 2) The document processing system of claim 2 wherein the feature extractor further converts location of each object into a first vector measured from a top-left corner from the corresponding filtered page to center of the object and into a second vector measured from a bottom-left corner from the corresponding filtered page to center of the object and selects the first or the second vector as a function of which of the first or the second vector has a larger magnitude; the feature extractor further converts location of each object into a third vector measured from a top-left corner from the corresponding filtered page to center of the object and into a fourth vector measured from a bottom-left corner from the corresponding filtered page to center of the object and selects the third or the fourth vector as a function of which of the first or the second vector has a larger magnitude; and identifies the selected vector among the first and second vectors corresponding to each object and identifies the selected vector among the third and fourth vectors corresponding to each object and selects a vector from the selected vector among the first and second vectors corresponding to each object and from the selected vector among the third and fourth vectors corresponding to each object. ### Claim 9 A tangible storage medium, having stored thereupon one or more program modules comprising computer-executable instructions for execution on a computer system, the computer-executable instructions causing the computer system to implement a document processing system that interacts with a data storage for storing a plurality of documents in image format, the documents comprising one or more sets of documents, each set of documents comprising documents of having similar formatting; the computer system executing one or more of the program modules that when executed implement: an object recognizer that processes documents in a set of documents and generates for each document in the set of documents, a set of page objects, wherein the page objects are organized in a two-dimensional arrangement in a corresponding document; a feature extractor responsive to a set of page objects for extracting known features from each set of page objects by sequentially organizing each object in each set of page objects into a one-dimensional array, each object positioned in a corresponding one-dimensional array as a function of location in the corresponding set of page objects; a template identifier for processing each one-dimensional array by comparing each one-dimensional array to a plurality of known one-dimensional arrays, each of the known one-dimensional arrays corresponding to an image encoded document having a known formatting; and classifying the document in image format corresponding to the one-dimensional array into a classification having a known formatting upon a match within a predetermined match threshold. ### Claim 10 (depends on 9) The tangible storage medium of claim 9 wherein the feature extractor sequentially organizes each object in each filtered page into a one-dimensional array as a function of location of each object from top-to-bottom and then left-to-right in the corresponding filtered page. ### Claim 11 (depends on 10) The tangible storage medium of claim 10 wherein the feature extractor further converts location of each object into a first vector measured from a top-right corner from the corresponding filtered page to center of the object and into a second vector measured from a bottom-right corner from the corresponding filtered page to center of the object and selects the first or the second vector as a function of which of the first or the second vector has a larger magnitude. ### Claim 12 (depends on 9) The tangible storage medium of claim 9 wherein the object recognizer comprises a deep neural network trained to identify objects in at least one of the sets of documents and to generate for each document processed by the deep neural network, a set of page objects. ### Claim 13 (depends on 9) The tangible storage medium of claim 9 wherein the object recognizer comprises an optical character recognizer that generates for each document processed by the optical character recognizer, a set of page objects. ### Claim 14 (depends on 9) The tangible storage medium of claim 9 further comprising a higher order object filter for processing each set of page objects organized in a two dimensional arrangement to generate for each set of page objects a filtered page by removing lower order objects from the set of page objects and wherein the filtered page is provided to the feature extractor as the set of page objects. ### Claim 15 (depends on 10) The tangible storage medium of claim 10 wherein the feature extractor further converts location of each object into a first vector measured from a top-left corner from the corresponding filtered page to center of the object and into a second vector measured from a bottom-left corner from the corresponding filtered page to center of the object and selects the first or the second vector as a function of which of the first or the second vector has a larger magnitude. ### Claim 16 (depends on 10) The tangible storage medium of claim 10 wherein the feature extractor further converts location of each object into a first vector measured from a top-left corner from the corresponding filtered page to center of the object and into a second vector measured from a bottom-left corner from the corresponding filtered page to center of the object and selects the first or the second vector as a function of which of the first or the second vector has a larger magnitude; the feature extractor further converts location of each object into a third vector measured from a top-left corner from the corresponding filtered page to center of the object and into a fourth vector measured from a bottom-left corner from the corresponding filtered page to center of the object and selects the third or the fourth vector as a function of which of the first or the second vector has a larger magnitude; and identifies the selected vector among the first and second vectors corresponding to each object and identifies the selected vector among the third and fourth vectors corresponding to each object and selects a vector from the selected vector among the first and second vectors corresponding to each object and from the selected vector among the third and fourth vectors corresponding to each object.