Back to results
US 11,238,351 B2Granted
Grading sources and managing evidence for intelligence analysis
AssigneeInternational Business Machines Corporation
InventorsDarryl M. Adderly, Corville O. Allen, Robert K. Tucker
Life of this patent
Estimating the term…
Abstract
Mechanisms for evaluating an evidential statement in a corpus of evidence are provided. An evidential statement is received for determining a level of confidence in a hypothetical ontological link of an ontology. A source of the evidential statement is identified and a grading of the source of the evidential statement is determined based on a source grading measurement value indicative of a degree of reliability and credibility of the source. An indication of trustworthiness of the evidential statement is generated based on the source grading measurement value. A representation of the indication of trustworthiness of the evidential statement is output in association with the evidential statement.
Claims20
A method, in a data processing system comprising a processor and a memory, for evaluating an evidential statement in a corpus of evidence, comprising:
- identifying, by the corpora management engine executing on the data processing system, a source of an evidential statement in a corpus of electronically stored evidence data, based on a source profile directory data structure associated with the corpus of electronically stored evidence data;
- determining, by the corpora management engine executing on the data processing system, a grading of the source of the evidential statement based on a source grading measurement value indicative of a degree of reliability and credibility of the source calculated based on information in the source profile directory data structure corresponding to the source;
- generating, by the corpora management engine executing on the data processing system, an indication of trustworthiness of the evidential statement based on the source grading measurement value;
- modifying, by the corpora management engine executing on the data processing system, electronically stored evidence data of the corpus based on the grading of the source of the evidential statement, at least by one of increasing the amount of evidence data from the source that is in the corpus or decreasing the amount of evidence data from the source in the corpus based on the indication of trustworthiness of the evidential statement, to thereby generate a modified corpus;
- and
- automatically processing an input natural language question by an artificial intelligence question answering computer system at least by automatically generating candidate answers to the input natural language question by extracting the candidate answers from the modified corpus and evaluating evidential support for the candidate answers based on the evidence data in the modified corpus to rank the candidate answers and generate a final answer to the input natural language question based on the ranking of the candidate answers.
3 more elements
A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a data processing system, causes the data processing system to:
- identify, by the corpora management engine executing on the data processing system, a source of an evidential statement in a corpus of electronically stored evidence data, based on a source profile directory data structure associated with the corpus of electronically stored evidence data;
- determine, by the corpora management engine executing on the data processing system, a grading of the source of the evidential statement based on a source grading measurement value indicative of a degree of reliability and credibility of the source calculated based on information in the source profile directory data structure corresponding to the source;
- generate, by the corpora management engine executing on the data processing system, an indication of trustworthiness of the evidential statement based on the source grading measurement value;
- modify, by the corpora management engine executing on the data processing system, electronically stored evidence data of the corpus based on the grading of the source of the evidential statement, at least by one of increasing the amount of evidence data from the source that is in the corpus or decreasing the amount of evidence data from the source in the corpus based on the indication of trustworthiness of the evidential statement, to thereby generate a modified corpus;
- and
- automatically process an input natural language question by an artificial intelligence question answering computer system at least by automatically generating candidate answers to the input natural language question by extracting the candidate answers from the modified corpus and evaluating evidential support for the candidate answers based on the evidence data in the modified corpus to rank the candidate answers and generate a final answer to the input natural language question based on the ranking of the candidate answers.
3 more elements
An apparatus comprising:
- at least one processor;
- and
- at least one memory coupled to the at least one processor, wherein the at least one memory comprises instructions which, when executed by the at least one processor, cause the at least one processor to:
- identify, by the corpora management engine executing on the at least one processor, a source of an evidential statement in a corpus of electronically stored evidence data, based on a source profile directory data structure associated with the corpus of electronically stored evidence data;
- determine, by the corpora management engine executing on the at least one processor, a grading of the source of the evidential statement based on a source grading measurement value indicative of a degree of reliability and credibility of the source calculated based on information in the source profile directory data structure corresponding to the source;
- generate, by the corpora management engine executing on the at least one processor, an indication of trustworthiness of the evidential statement based on the source grading measurement value;
- modify, by the corpora management engine executing on the at least one processor, electronically stored evidence data of the corpus based on the grading of the source of the evidential statement, at least by one of increasing the amount of evidence data from the source that is in the corpus or decreasing the amount of evidence data from the source in the corpus based on the indication of trustworthiness of the evidential statement, to thereby generate a modified corpus;
- and
- automatically process an input natural language question by an artificial intelligence question answering computer system at least by automatically generating candidate answers to the input natural language question by extracting the candidate answers from the modified corpus and evaluating evidential support for the candidate answers based on the evidence data in the modified corpus to rank the candidate answers and generate a final answer to the input natural language question based on the ranking of the candidate answers.
6 more elements
Dependencies are inferred from the claim text.