# US 10,482,180 B2 · Generating ground truth for questions based on data found in structured resources > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,482,180 B2 - Document ID: US10482180B2 - Status: Granted - Kind: B2 - Application number: 15/816,089 - Filed: 2017-11-17 - Published: 2019-05-23 - Granted: 2019-11-19 - Earliest claimed: 2017-11-17 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Priscilla Santos Moraes; Kathryn V. Banks; Dan G. Tecuci - Web page: https://patentgrep.com/patent/US10482180B2 ## CPC - G06F 40/30: Semantic analysis - G06F 16/3329: Natural language query formulation - G06F 16/3344: {using natural language analysis} - G06F 40/186: Templates - G06F 40/279: Recognition of textual entities - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Ground truth for a cognitive system is generated from a structured resource such as a table by identifying a subject of the structured resource and field headers. Linguistic analysis is performed on a given header to establish an interrogative context, and a question is generated relating to the subject based on the interrogative context, including an implementation of one or more mathematical operators. The question is generated using a question template, and has a question phrase based on the interrogative context, an operator phrase based on the selected operator, and a keyword phrase based on the subject. An answer to the question is determined by carrying out a computation that applies the selected operator(s) to one or more of the data values, to form a question-and-answer pair that is added to the ground truth. A filtering step is preferably used to ensure that the question-and-answer pair is valid. ## Claims ### Claim 1 A method of providing ground truth for a cognitive system comprising: receiving a structured resource having a set of data values, by executing first instructions in a computer system; receiving a set of operators, by executing second instructions in the computer system; identifying a subject of the structured resource and at least one field header of the structured resource, by executing third instructions in the computer system; performing linguistic analysis on the field header to determine an interrogative context, by executing fourth instructions in the computer system; generating at least one question relating to the subject based on the interrogative context wherein the question includes an implementation of a selected one of the operators, by executing fifth instructions in the computer system; and determining an answer for the question to form a question-and-answer pair. ### Claim 2 (depends on 1) The method of claim 1 wherein the question includes a question phrase based on the interrogative context, an operator phrase based on the selected operator, and a keyword phrase based on the subject. ### Claim 3 (depends on 1) The method of claim 1 wherein the data values are numerical values and the operators are mathematical operators. ### Claim 4 (depends on 1) The method of claim 1 wherein the question is generated using a question template. ### Claim 5 (depends on 1) The method of claim 1 wherein said determining the answer includes carrying out a computation by applying the selected operator to one or more of the data values. ### Claim 6 (depends on 1) The method of claim 1 further comprising determining that the question-and-answer pair is valid. ### Claim 7 (depends on 1) The method of claim 1 further comprising: storing the question-and-answer pair as part of the ground truth for the cognitive system; and using the cognitive system to formulate a response to a natural language query. ### Claim 8 A computer system comprising: one or more processors which process program instructions; a memory device connected to said one or more processors; and program instructions residing in said memory device for providing ground truth to a cognitive system by receiving a structured resource having a set of data values, receiving a set of operators, identifying a subject of the structured resource and at least one field header of the structured resource, performing linguistic analysis on the field header to determine an interrogative context, generating at least one question relating to the subject based on the interrogative context wherein the question includes an implementation of a selected one of the operators, and determining an answer for the question to form a question-and-answer pair. ### Claim 9 (depends on 8) The computer system of claim 8 wherein the question includes a question phrase based on the interrogative context, an operator phrase based on the selected operator, and a keyword phrase based on the subject. ### Claim 10 (depends on 8) The computer system of claim 8 wherein the data values are numerical values and the operators are mathematical operators. ### Claim 11 (depends on 8) The computer system of claim 8 wherein the question is generated using a question template. ### Claim 12 (depends on 8) The computer system of claim 8 wherein determining the answer includes carrying out a computation by applying the selected operator to one or more of the data values. ### Claim 13 (depends on 8) The computer system of claim 8 wherein said program instructions further determine that the question-and-answer pair is valid. ### Claim 14 (depends on 8) The computer system of claim 8 wherein said program instructions further store the question-and-answer pair as part of the ground truth for the cognitive system. ### Claim 15 A computer program product comprising: a computer readable storage medium; and program instructions residing in said storage medium for providing ground truth to a cognitive system by receiving a structured resource having a set of data values, receiving a set of operators, identifying a subject of the structured resource and at least one field header of the structured resource, performing linguistic analysis on the field header to determine an interrogative context, generating at least one question relating to the subject based on the interrogative context wherein the question includes an implementation of a selected one of the operators, and determining an answer for the question to form a question-and-answer pair. ### Claim 16 (depends on 15) The computer program product of claim 15 wherein the question includes a question phrase based on the interrogative context, an operator phrase based on the selected operator, and a keyword phrase based on the subject. ### Claim 17 (depends on 15) The computer program product of claim 15 wherein the data values are numerical values and the operators are mathematical operators. ### Claim 18 (depends on 15) The computer program product of claim 15 wherein the question is generated using a question template. ### Claim 19 (depends on 15) The computer program product of claim 15 wherein determining the answer includes carrying out a computation by applying the selected operator to one or more of the data values. ### Claim 20 (depends on 15) The computer program product of claim 15 wherein said program instructions further determine that the question-and-answer pair is valid.