System and method for domain adaptation in question answering

A method for providing adaptation to a question answering system, wherein the question answering system has associated therewith a plurality of trace data and a question-answer set, the method comprising the steps of:

• submitting a set of questions to the question answering system;
• receiving back from the question answering system a set of answers generated in response to the set of questions;
• comparing the set of answers received back from the question answering system to answers in the question-answer set;
• and
• generating, based on the comparison and the plurality of trace data, an estimate of how much more training data is needed for each of a plurality of question types or answer types;
• wherein the training data comprises a plurality of training questions along with a corresponding plurality of correct training answers;
• wherein the trace data is generated during the generation of the set of answers;
• and
• wherein the step of generating further comprises the following steps:
• (a) for each question type, successively sample an increasing number of question-answer pairs from the question-answer set;
• (b) automatically train the question answering system using the sampled question-answer pairs;
• (c) automatically compute a function relating the question answering system performance on the remaining questions relative to the number of successively sampled question-answer pairs;
• and
• (d) extrapolate from the function a number of training question-answer pairs of each question type.