# US 2019/0171704 A1 · Heuristic Domain Targeted Table Detection and Extraction Technique > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0171704 A1 - Document ID: US20190171704A1 - Status: Published application - Kind: A1 - Application number: 15/829,118 - Filed: 2017-12-01 - Published: 2019-06-06 - Earliest claimed: 2017-12-01 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Jean-François Buisson; Keith D. Cramer - Web page: https://patentgrep.com/patent/US20190171704A1 ## CPC - G06F 16/258: {Data format conversion from or to a database} - G06F 16/3325: {Reformulation based on results of preceding query} - G06F 40/131: Fragmentation of text files, e.g. creating reusable text-blocks; Linking to fragments, e.g. using XInclude; Namespaces - G06F 40/163: Handling of whitespace - G06F 40/174: Form filling; Merging - G06F 40/177: of tables; using ruled lines - G06V 30/412: Layout analysis of documents structured with printed lines or input boxes, e.g. business forms or tables - G06V 30/414: Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text - G06V 30/10: Character recognition - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A method, system, and apparatus are provided for processing tables embedded within documents wherein a first table header is detected by using semantic groupings of table header terms to identify a minimum number of table header terms in a scanned line of an text document; a potential data zone is extracted by applying white space correlation analysis to a portion of the text document that is adjacent to the first table header; one or more data zone columns from the potential data zone are grouped and aligned with a corresponding header column in the first table header to form a candidate table; data cleansing is performed on the candidate table; and then one or more columns of the candidate table are evaluated using natural language processing to apply a specified table analysis. ## Claims ### Claim 1 A method, in an information handling system comprising a processor and a memory, for processing tables embedded within documents, the method comprising: receiving, by the system, an input document comprising embedded table data; converting, by the system, the input document to a text document comprising table data; detecting, by the system, a first table header by using semantic groupings of table header terms to identify a minimum number of table header terms in a scanned line of the text document; extracting, by the system, a potential data zone by applying white space correlation analysis to a portion of the text document that is adjacent to the first table header; grouping, by the system, one or more data zone columns from the potential data zone; aligning, by the system, each of the one or more data zone columns with a corresponding header column in the first table header to form a candidate table; performing, by the system, data cleansing on the candidate table; and evaluating, by the system, one or more columns of the candidate table using natural language processing to apply a specified table analysis. ### Claim 2 (depends on 1) The method of claim 1, where the embedded table data in the input document comprises embedded medical data. ### Claim 3 (depends on 1) The method of claim 1, where the input document comprises a pdf image file, and where converting the input document comprises transforming the pdf image file into the text document with Optical Character Reading (OCR) software. ### Claim 4 (depends on 1) The method of claim 1, where the detecting the first table header comprises: comparing, by the system, each word of the scanned line to an XML file defining a plurality of candidate column header values that are semantically grouped into separate column categories to detect a table header term if the word matches a candidate header value; and creating, by the system, the first table header if at least two table header terms detected in the scanned line. ### Claim 5 (depends on 4) The method of claim 4, where comparing each word of the scanned line to the XML file comprises using a dictionary based search with fuzzy matching to detect the table header term. ### Claim 6 (depends on 1) The method of claim 1, where grouping one or more data zone columns comprises grouping tokenized words in each line of the potential data zone into corresponding data zone columns and storing an average x location for each data zone column. ### Claim 7 (depends on 6) The method of claim 6, where aligning each of the one or more data zone columns comprises aligning the average x location for each data zone column with an x location for the corresponding header column in the first table header. ### Claim 8 (depends on 1) The method of claim 1, where performing data cleansing on the candidate table comprises removing table comments and trailing remnant lines from the end of the candidate table. ### Claim 9 (depends on 1) The method of claim 1, evaluating one or more columns comprises validating an abnormal value indication from a first column of the candidate table by comparing a corresponding result value from a second column of the candidate table with a corresponding range value from a third column of the candidate table to detect if the result value fits within the range value. ### Claim 10 An information handling system comprising: one or more processors; a memory coupled to at least one of the processors; a set of instructions stored in the memory and executed by at least one of the processors to process tables embedded within documents, wherein the set of instructions are executable to perform actions of: receiving, by the system, an input document comprising embedded table data; converting, by the system, the input document to a text document comprising table data; detecting, by the system, a first table header by using semantic groupings of table header terms to identify a minimum number of table header terms in a scanned line of the text document; extracting, by the system, a potential data zone by applying white space correlation analysis to a portion of the text document that is adjacent to the first table header; grouping, by the system, one or more data zone columns from the potential data zone; aligning, by the system, each of the one or more data zone columns with a corresponding header column in the first table header to form a candidate table; performing, by the system, data cleansing on the candidate table; and evaluating, by the system, one or more columns of the candidate table using natural language processing to apply a specified table analysis. ### Claim 11 (depends on 10) The information handling system of claim 10, wherein the set of instructions are executable to detect the first table header by: comparing, by the system, each word of the scanned line to an XML file defining a plurality of candidate column header values that are semantically grouped into separate column categories to detect a table header term if the word matches a candidate header value; and creating, by the system, the first table header if at least two table header terms detected in the scanned line. ### Claim 12 (depends on 11) The information handling system of claim 11, wherein the set of instructions are executable to compare each word of the scanned line to the XML file by using a dictionary based search with fuzzy matching to detect the table header term. ### Claim 13 (depends on 10) The information handling system of claim 10, wherein the set of instructions are executable to group one or more data zone columns by grouping tokenized words in each line of the potential data zone into corresponding data zone columns and storing an average x location for each data zone column. ### Claim 14 (depends on 13) The information handling system of claim 13, wherein the set of instructions are executable to align each of the one or more data zone columns by aligning the average x location for each data zone column with an x location for the corresponding header column in the first table header. ### Claim 15 (depends on 10) The information handling system of claim 10, wherein the set of instructions are executable to perform data cleansing by removing table comments and trailing remnant lines from the end of the candidate table. ### Claim 16 (depends on 10) The information handling system of claim 10, wherein the set of instructions are executable to evaluating one or more columns by validating an abnormal value indication from a first column of the candidate table by comparing a corresponding result value from a second column of the candidate table with a corresponding range value from a third column of the candidate table to detect if the result value fits within the range value. ### Claim 17 A computer program product stored in a computer readable storage medium, comprising computer instructions that, when executed by an information handling system, causes the system to process tables embedded within documents by: receiving, by the system, an input document comprising embedded table data; converting, by the system, the input document to a text document comprising table data; detecting, by the system, a first table header by using semantic groupings of table header terms to identify a minimum number of table header terms in a scanned line of the text document; extracting, by the system, a potential data zone by applying white space correlation analysis to a portion of the text document that is adjacent to the first table header; grouping, by the system, one or more data zone columns from the potential data zone; aligning, by the system, each of the one or more data zone columns with a corresponding header column in the first table header to form a candidate table; performing, by the system, data cleansing on the candidate table; and evaluating, by the system, one or more columns of the candidate table using natural language processing to apply a specified table analysis. ### Claim 18 (depends on 17) The computer program product of claim 17, further comprising computer instructions that, when executed by an information handling system, causes the system to detect the first table header by: comparing, by the system, each word of the scanned line to an XML file defining a plurality of candidate column header values that are semantically grouped into separate column categories by using a dictionary based search with fuzzy matching to detect a table header term if the word matches a candidate header value; and creating, by the system, the first table header if at least two table header terms detected in the scanned line. ### Claim 19 (depends on 18) The computer program product of claim 18, further comprising computer instructions that, when executed by an information handling system, causes the system to group one or more data zone columns by grouping tokenized words in each line of the potential data zone into corresponding data zone columns and storing an average x location for each data zone column. ### Claim 20 (depends on 19) The computer program product of claim 19, further comprising computer instructions that, when executed by an information handling system, causes the system to align each of the one or more data zone columns by aligning the average x location for each data zone column with an x location for the corresponding header column in the first table header.