# US 8,280,838 B2 · Evidence evaluation system and method based on question answering

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 8,280,838 B2
- Document ID: US8280838B2
- Status: Granted
- Kind: B2
- Application number: 12/561,802
- Filed: 2009-09-17
- Published: 2011-03-17
- Granted: 2012-10-02
- Earliest claimed: 2009-09-17
- Assignees: International Business Machines Corporation
- Inventors: David A. Ferrucci; Wlodek W. Zadrozny
- Web page: https://patentgrep.com/patent/US8280838B2

## CPC

- G06N 5/04: Inference or reasoning models
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

An evidence evaluation method and system based on question answering converts a report of analyzed information and/or a model of information into a collection of questions, determines answers for the collection of questions. A fact in the report is marked as being supported if one or more of the answers for the collection of questions support the fact. A fact in the report of analyzed facts is marked as being refuted if one or more of the answers for the collection of questions refute the fact. The method and system also may collect the answers as evidence and add the evidence to the model of information to create an updated model of information. The steps may be repeated using the updated report and updated model.

## Claims

### Claim 1

An evidence evaluation method based on question answering, comprising:
converting information into a collection of questions by a processor;
determining answers to the collection of questions;
marking a fact in the information as being supported if one or more of the answers to the collection of questions support the fact;
marking a fact in the information as being refuted if one or more of the answers to the collection of questions refute the fact;
collecting evidence that support the answers; and
adding the evidence to a model of information to create an updated model of information.

### Claim 2 (depends on 1)

The method of claim 1, wherein the information includes a report prepared from analyzing data.

### Claim 3 (depends on 1)

The method of claim 1, wherein the information includes the model of information.

### Claim 4 (depends on 1)

The method of claim 1, wherein the model of information includes structured data.

### Claim 5 (depends on 2)

The method of claim 2, wherein the report includes natural language text.

### Claim 6 (depends on 1)

The method of claim 1, wherein the step of marking a fact in the information as being supported further includes annotating the fact with metadata describing supporting answer for the fact.

### Claim 7 (depends on 1)

The method claim 1, wherein the evidence include lack of support if one or more of the answers to the collection of questions refute the fact.

### Claim 8 (depends on 1)

The method of claim 1, wherein an updated report is created by analyzing the updated model.

### Claim 9 (depends on 8)

The method of claim 8, wherein the steps of converting, determining, marking, collecting and adding are repeated with the updated report.

### Claim 10 (depends on 1)

The method of claim 1, wherein the steps of converting, determining, marking, collecting and adding are repeated with the updated model.

### Claim 11 (depends on 1)

The method of claim 1, wherein the steps are performed as a background process as analysis is performed to create the information.

### Claim 12

A system for an evidence evaluation method based on question answering, comprising:
an editor operable to allow entering of information;
an editor plug-in operable to automatically convert the information into a collection of questions;
question answering processing module operable to determine answers for the collection of questions;
said editor plug-in further operable to mark a fact in the information as being supported if one or more of the answers for the collection of questions support the fact and marking a fact in the information as being refuted if one or more of the answers for the collection of questions refute the fact, said editor plug-in further operable to collect the answers as evidence and add the evidence to the model of information to create an updated model of information.

### Claim 13 (depends on 12)

The system of claim 12, wherein the information includes a report.

### Claim 14 (depends on 12)

The system of claim 12, wherein the information includes a model of information, the model including structured data.

### Claim 15 (depends on 12)

The system of claim 12, further including:
reasoning and representation processing module operable to convert external data into structured data, wherein the question answering processing module uses the converted external data to determine answers to the collection of questions.

### Claim 16 (depends on 15)

The system of claim 15, wherein the external data includes data from one or more of an information database, or Internet, or combinations thereof.

### Claim 17 (depends on 12)

The system of claim 12, wherein the updated model of information is used to further as the information for forming a collection of questions.

### Claim 18

A program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine to perform an evidence evaluation method based on question answering, comprising:
converting information into a collection of questions by a processor;
determining answers to the collection of questions;
marking a fact in the information as being supported if one or more of the answers to the collection of questions support the fact;
marking a fact in the information as being refuted if one or more of the answers to the collection of questions refute the fact;
collecting evidence that support the answers; and
adding the evidence to a model of information to create an updated model of information.

### Claim 19 (depends on 18)

The program storage device of claim 18, wherein the step of marking a fact in the information as being supported further includes annotating the fact with metadata describing supporting answer for the fact.

### Claim 20 (depends on 1)

The program storage device claim 1, wherein the evidence includes lack of support if one or more of the answers to the collection of questions refute the fact.
