# US 2018/0025075 A1 · Evaluating Temporal Relevance in Question Answering > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2018/0025075 A1 - Document ID: US20180025075A1 - Status: Published application - Kind: A1 - Application number: 15/214,529 - Filed: 2016-07-20 - Published: 2018-01-25 - Earliest claimed: 2016-07-20 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Charles E. Beller; Paul J. Chase, JR.; Richard L. Darden; Edward G. Katz - Web page: https://patentgrep.com/patent/US20180025075A1 ## CPC - G06F 40/284: Lexical analysis, e.g. tokenisation or collocates - G06F 16/3329: Natural language query formulation - G06F 40/30: Semantic analysis - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Mechanisms are provided for evaluating a temporal relevance of a candidate answer to an input natural language question. An input natural language question is received and analyzed to identify a temporal focus of the input natural language question. A corpus of documents is processed based on the input natural language question to generate candidate answers to the input natural language question, where each candidate answer is processed to identify at least one contextual temporal focus associated with the candidate answer. The at least one contextual temporal focus is compared with the temporal focus of the input natural language question and a measure of temporal relevance of the candidate answer based on results of the comparison is generated. A final answer to the input natural language question is output based on the measure of temporal relevance. ## Claims ### Claim 1 A method, in a data processing system comprising at least one processor and a memory, the memory comprising instructions which are executed by the at least one processor to specifically configure the at least one processor to implement a Question Answering (QA) system, the method comprising: receiving, by the QA system from another computing device, an input natural language question for answering by the QA system; analyzing, by the QA system, the input natural language question to identify a temporal focus of the input natural language question; processing, by the QA system, a corpus of documents based on the input natural language question to generate candidate answers to the input natural language question; processing, by the QA system, each candidate answer, to identify at least one contextual temporal focus associated with the candidate answer; comparing, by the QA system, the at least one contextual temporal focus with the temporal focus of the input natural language question; generating, by the QA system, a measure of temporal relevance of the candidate answer based on results of the comparison; and outputting, by the QA system to the computing device, a final answer to the input natural language question based on the measure of temporal relevance. ### Claim 2 (depends on 1) The method of claim 1, further comprising analyzing the documents of the corpus, wherein analyzing the documents of the corpus comprises, for each document: determining a document relevance datetime of the document based on at least one of metadata or content of the document, wherein the document relevance datetime applies to all content of the document; and associating, with each definite temporal expression in content of the document, a datetime with the definite temporal expression, and wherein the at least one contextual temporal focus of a candidate answer generated based on a definite temporal expression in the content of the document is identified based on at least one of the document relevance datetime or the datetime of the definite temporal expression. ### Claim 3 (depends on 2) The method of claim 2, wherein analyzing the documents of the corpus further comprises, for each document: associating with each token in the content of the document, a datetime based on a document relevance datetime associated with the document and a datetime associated with a definite temporal expression that is closest to the token in the document, and wherein the at least one contextual temporal focus of the candidate answer generated based on the definite temporal expression in the content of the document is identified based on the datetime of tokens in the candidate answer. ### Claim 4 (depends on 3) The method of claim 3, wherein the at least one contextual temporal focus of the candidate answer is identified based on the datetime of tokens in the candidate answer by combining datetimes of the tokens in the candidate answer according to a predetermined relationship. ### Claim 5 (depends on 4) The method of claim 4, wherein the predetermined relationship is one of a union of datetimes of tokens in the candidate answer or a minimally overlapping datetime evaluation. ### Claim 6 (depends on 2) The method of claim 2, wherein determining a document relevance datetime of the document comprises determining the document relevance datetime in accordance with an established prioritization of a plurality of types of relevance datetimes. ### Claim 7 (depends on 2) The method of claim 2, wherein the analyzing of the documents is done as part of an ingestion operation for ingesting the corpus prior to receipt of the input natural language question. ### Claim 8 (depends on 1) The method of claim 1, wherein analyzing the input natural language question to identify a temporal focus of the input natural language question comprises determining a current datetime and normalizing definite temporal expressions in the input natural language question relative to the current datetime. ### Claim 9 (depends on 1) The method of claim 1, wherein comparing the at least one contextual temporal focus with the temporal focus of the input natural language question comprises generating a score for the candidate answer based on a determined amount of overlap of the at least one contextual temporal focus and the temporal focus of the input natural language question. ### Claim 10 (depends on 1) The method of claim 1, wherein comparing the at least one contextual temporal focus with the temporal focus of the input natural language question comprises generating a score for the candidate answer based on a determined temporal proximity between the at least one contextual temporal focus and the temporal focus of the input natural language question. ### Claim 11 A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a computing device, causes the computing device to: receive an input natural language question for answering by the QA system; analyze the input natural language question to identify a temporal focus of the input natural language question; process a corpus of documents based on the input natural language question to generate candidate answers to the input natural language question; process each candidate answer to identify at least one contextual temporal focus associated with the candidate answer; compare the at least one contextual temporal focus with the temporal focus of the input natural language question; generate a measure of temporal relevance of the candidate answer based on results of the comparison; and output a final answer to the input natural language question based on the measure of temporal relevance. ### Claim 12 (depends on 11) The computer program product of claim 11, wherein the computer readable program further causes the computing device to analyze the documents of the corpus, wherein analyzing the documents of the corpus comprises, for each document: determining a document relevance datetime of the document based on at least one of metadata or content of the document, wherein the document relevance datetime applies to all content of the document; and associating, with each definite temporal expression in content of the document, a datetime with the definite temporal expression, and wherein the at least one contextual temporal focus of a candidate answer generated based on a definite temporal expression in the content of the document is identified based on at least one of the document relevance datetime or the datetime of the definite temporal expression. ### Claim 13 (depends on 12) The computer program product of claim 12, wherein analyzing the documents of the corpus further comprises, for each document: associating with each token in the content of the document, a datetime based on a document relevance datetime associated with the document and a datetime associated with a definite temporal expression that is closest to the token in the document, and wherein the at least one contextual temporal focus of the candidate answer generated based on the definite temporal expression in the content of the document is identified based on the datetime of tokens in the candidate answer. ### Claim 14 (depends on 13) The computer program product of claim 13, wherein the at least one contextual temporal focus of the candidate answer is identified based on the datetime of tokens in the candidate answer by combining datetimes of the tokens in the candidate answer according to a predetermined relationship. ### Claim 15 (depends on 14) The computer program product of claim 14, wherein the predetermined relationship is one of a union of datetimes of tokens in the candidate answer or a minimally overlapping datetime evaluation. ### Claim 16 (depends on 12) The computer program product of claim 12, wherein the computer readable program further causes the computing device to determine a document relevance datetime of the document at least by determining the document relevance datetime in accordance with an established prioritization of a plurality of types of relevance datetimes. ### Claim 17 (depends on 12) The computer program product of claim 12, wherein the analyzing of the documents is done as part of an ingestion operation for ingesting the corpus prior to receipt of the input natural language question. ### Claim 18 (depends on 11) The computer program product of claim 11, wherein the computer readable program further causes the computing device to analyze the input natural language question to identify a temporal focus of the input natural language question at least by determining a current datetime and normalizing definite temporal expressions in the input natural language question relative to the current datetime. ### Claim 19 (depends on 11) The computer program product of claim 11, wherein the computer readable program further causes the computing device to compare the at least one contextual temporal focus with the temporal focus of the input natural language question at least by generating a score for the candidate answer based on a determined temporal proximity between the at least one contextual temporal focus and the temporal focus of the input natural language question. ### Claim 20 An apparatus comprising: a processor; and a memory coupled to the processor, wherein the memory comprises instructions which, when executed by the processor, cause the processor to: receive an input natural language question for answering by the QA system; analyze the input natural language question to identify a temporal focus of the input natural language question; process a corpus of documents based on the input natural language question to generate candidate answers to the input natural language question; process each candidate answer to identify at least one contextual temporal focus associated with the candidate answer; compare the at least one contextual temporal focus with the temporal focus of the input natural language question; generate a measure of temporal relevance of the candidate answer based on results of the comparison; and output a final answer to the input natural language question based on the measure of temporal relevance.