Determining comprehensiveness of question paper given syllabus

A method, in a data processing system, for determining comprehensiveness of a question paper given a syllabus of topics, the method comprising:

• finding, by an answer and evidence generator of a question answering system executing on the data processing system, one or more answers based on the syllabus of topics for each question in the question paper;
• identifying, by the answer and evidence generator, evidence for the one or more answers in the syllabus for each question in the question paper;
• identifying, by a concept identifier of the question answering system, a set of concepts in the syllabus corresponding to the evidence for each question in the question paper to form a plurality of sets of concepts;
• and
• determining a value for a comprehensiveness metric for the question paper with respect to the syllabus of topics based on the plurality of sets of concepts, wherein the comprehensiveness metric comprises a coverage metric, wherein determining the value for the comprehensiveness metric comprises determining a union of the plurality of sets of concepts and determining a value of the coverage metric for the question paper to be equal to a length of the union of the plurality of sets of concepts divided by a length of the syllabus to form a result.