# US 12,112,562 B2 · Pre-processing a table in a document for natural language processing > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,112,562 B2 - Document ID: US12112562B2 - Status: Granted - Kind: B2 - Application number: 18/518,279 - Filed: 2023-11-22 - Published: 2024-03-21 - Granted: 2024-10-08 - Earliest claimed: 2021-01-21 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Scott Carrier; Ritwik Ray; Jonathan Chapin Rand; Jothilakshmi Sirangimoorthy; Hui Wang; Robert Fredenburg - Web page: https://patentgrep.com/patent/US12112562B2 ## CPC - G06F 40/30: Semantic analysis - G06V 30/412: Layout analysis of documents structured with printed lines or input boxes, e.g. business forms or tables - G06F 3/048: Interaction techniques based on graphical user interfaces [GUI] - G06F 3/0482: Interaction with lists of selectable items, e.g. menus - G06F 40/237: Lexical tools - G06F 40/40: Processing or translation of natural language (natural language analysis G06F40/20; semantic analysis G06F40/30) - G06V 30/416: Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Provided are a computer program product, system, and method for pre-processing a table in a document for natural language processing (NLP). A graphical user interface (GUI) provides a representation of table items in a table in a document including a set of a main element comprising an entity whose value is to be extracted, a conditional element that refines the entity, and a value element comprising a value for the entity. Graphical controls are rendered in the GUI to enable a user to select an element from the table to be the main element, conditional element, and value element. The set of the main element, conditional element, and value element are updated with the user selected element to form a modified set. The modified set of the main element, conditional element, and the value element are provided to an NLP engine to perform natural language processing. ## Claims ### Claim 1 A computer program product for natural language processing (NLP) of a document including a table, wherein the computer program product comprises a computer readable storage medium having computer readable program instructions executed by a processor to perform operations, the operations comprising: generating a graphical user interface (GUI) providing a representation of table items in a table in a document including a set of a main element comprising an entity whose value is to be extracted, a conditional element that refines the entity, and a value element comprising a value for the entity; rendering, in the GUI, graphical controls to enable a user to select an element from the table to be the main element, the conditional element, and the value element; updating the set of the main element, the conditional element, and the value element with the user selected element to be one of the main element, the conditional element, and the value element to form a modified set; providing the modified set of the main element, the conditional element, and the value element to an NLP engine to perform natural language processing of the document including the table, using the modified set of the main element, the conditional element, and the value element to interpret the table; rendering, in the GUI, graphical controls to enable a user to select a lexical connector between the main element and one of conditional elements; and updating the set to include in the updated set the user selected lexical connector. ### Claim 2 (depends on 1) The computer program product of claim 1, wherein the graphical controls enable the user to select the lexical connector between any pair of conditional elements and between a conditional element and a value element. ### Claim 3 (depends on 1) The computer program product of claim 1, wherein the operations further comprise: rendering, in the GUI, graphical controls to enable a user to add a conditional element and a lexical connector for the added conditional element; and updating the set to include in the updated set the added conditional element and the lexical connector for the added conditional element. ### Claim 4 (depends on 1) The computer program product of claim 1, wherein the operations further comprise: providing display screen coordinate mappings mapping display screen coordinates of a display screen in which the table is displayed to document coordinates in the document including the table; in response to receiving user selection of an area in the display screen, determine, from the display screen coordinate mappings, document coordinates mapping to the user selected area in the display screen; determining a table item in the document that maps to the determined document coordinates; rendering, in the GUI, a highlighting of the determined table item in the display screen; and indicating the determined table item as one of a main element, a conditional element or a value element in the updated set. ### Claim 5 (depends on 1) The computer program product of claim 1, wherein the operations further comprise: providing display screen coordinate mappings mapping display screen coordinates of a display screen in which the table is displayed to document coordinates in the document including the table; in response to receiving user selection of an area in the display screen, determine, from the display screen coordinate mappings, document coordinates mapping to the display screen coordinates; determining a table item in the document that maps to the document coordinates; and receiving user selection to designate the determined table item as one of a main element, a conditional element or a value element in the updated set. ### Claim 6 (depends on 5) The computer program product of claim 5, wherein the operations further comprise: updating the set to include in the updated set the determined table item as the main element, conditional element or value element the user selected. ### Claim 7 (depends on 5) The computer program product of claim 5, wherein the operations further comprise: in response to the determined table item contributing to a value, locating and rendering the value recognized by the NLP engine. ### Claim 8 A system for natural language processing (NLP) of a document including a table, comprising: a processor; and a computer readable storage medium having computer readable program instructions executed by a processor to perform operations, the operations comprising: generating a graphical user interface (GUI) providing a representation of table items in a table in a document including a set of a main element comprising an entity whose value is to be extracted, a conditional element that refines the entity, and a value element comprising a value for the entity; rendering, in the GUI, graphical controls to enable a user to select an element from the table to be the main element, the conditional element, and the value element; updating the set of the main element, the conditional element, and the value element with the user selected element to be one of the main element, the conditional element, and the value element to form a modified set; providing the modified set of the main element, the conditional element, and the value element to an NLP engine to perform natural language processing of the document including the table, using the modified set of the main element, the conditional element, and the value element to interpret the table; rendering, in the GUI, graphical controls to enable a user to select a lexical connector between the main element and one of conditional elements; and updating the set to include in the updated set the user selected lexical connector. ### Claim 9 (depends on 8) The system of claim 8, wherein the graphical controls enable the user to select the lexical connector between any pair of conditional elements and between a conditional element and a value element. ### Claim 10 (depends on 8) The system of claim 8, wherein the operations further comprise: rendering, in the GUI, graphical controls to enable a user to add a conditional element and a lexical connector for the added conditional element; and updating the set to include in the updated set the added conditional element and the lexical connector for the added conditional element. ### Claim 11 (depends on 8) The system of claim 8, wherein the operations further comprise: providing display screen coordinate mappings mapping display screen coordinates of a display screen in which the table is displayed to document coordinates in the document including the table; in response to receiving user selection of an area in the display screen, determine, from the display screen coordinate mappings, document coordinates mapping to the user selected area in the display screen; determining a table item in the document that maps to the determined document coordinates; rendering, in the GUI, a highlighting of the determined table item in the display screen; and indicating the determined table item as one of a main element, a conditional element or a value element in the updated set. ### Claim 12 (depends on 8) The system of claim 8, wherein the operations further comprise: providing display screen coordinate mappings mapping display screen coordinates of a display screen in which the table is displayed to document coordinates in the document including the table; in response to receiving user selection of an area in the display screen, determine, from the display screen coordinate mappings, document coordinates mapping to the display screen coordinates; determining a table item in the document that maps to the document coordinates; and receiving user selection to designate the determined table item as one of a main element, a conditional element or a value element in the updated set. ### Claim 13 A computer implemented method for natural language processing (NLP) of a document including a table, comprising: generating a graphical user interface (GUI) providing a representation of table items in a table in a document including a set of a main element comprising an entity whose value is to be extracted, a conditional element that refines the entity, and a value element comprising a value for the entity; rendering, in the GUI, graphical controls to enable a user to select an element from the table to be the main element, the conditional element, and the value element; updating the set of the main element, the conditional element, and the value element with the user selected element to be one of the main element, the conditional element, and the value element to form a modified set; providing the modified set of the main element, the conditional element, and the value element to an NLP engine to perform natural language processing of the document including the table, using the modified set of the main element, the conditional element, and the value element to interpret the table; rendering, in the GUI, graphical controls to enable a user to select a lexical connector between the main element and one of conditional elements; and updating the set to include in the updated set the user selected lexical connector. ### Claim 14 (depends on 13) The method of claim 13, wherein the graphical controls enable the user to select the lexical connector between any pair of conditional elements and between a conditional element and a value element. ### Claim 15 (depends on 13) The method of claim 13, further comprising: rendering, in the GUI, graphical controls to enable a user to add a conditional element and a lexical connector for the added conditional element; and updating the set to include in the updated set the added conditional element and the lexical connector for the added conditional element. ### Claim 16 (depends on 13) The method of claim 13, further comprising: providing display screen coordinate mappings mapping display screen coordinates of a display screen in which the table is displayed to document coordinates in the document including the table; in response to receiving user selection of an area in the display screen, determine, from the display screen coordinate mappings, document coordinates mapping to the user selected area in the display screen; determining a table item in the document that maps to the determined document coordinates; rendering, in the GUI, a highlighting of the determined table item in the display screen; and indicating the determined table item as one of a main element, a conditional element or a value element in the updated set. ### Claim 17 (depends on 13) The method of claim 13, further comprising: providing display screen coordinate mappings mapping display screen coordinates of a display screen in which the table is displayed to document coordinates in the document including the table; in response to receiving user selection of an area in the display screen, determine, from the display screen coordinate mappings, document coordinates mapping to the display screen coordinates; determining a table item in the document that maps to the document coordinates; and receiving user selection to designate the determined table item as one of a main element, a conditional element or a value element in the updated set.