Document type-specific quality model
AssigneeInternational Business Machines Corporation
InventorsBrien H. Muschett, Andrew R. Freed, Roberto Delima, David Contreras, Krishna Mahajan
Life of this patent
Estimating the term…
Abstract
An approach is provided that receives a document and a document type of the document. The document type identifies a document category to which the received document belongs. A set of linguistic metrics are retrieved that correspond to the document type. A quality of the received document is automatically determined based on a set of linguistic features found in the document as compared to the retrieved set of linguistic metrics. The document is then ingested into a corpus that is utilized by a question-answering (QA) system. The ingestion of the document is based on the determined quality.
Claims17
A method implemented by an information handling system that includes a processor and a memory accessible by the processor, the method comprising:
- receiving a document and a document type, wherein the document type identifies a document category to which the received document belongs;
- identifying a set of structural linguistic features found in the document, wherein the set of structural linguistic features are associated with the document type;
- retrieving a set of structural linguistic metrics corresponding to the document type;
- comparing the set of structural linguistic features found in the document to the retrieved set of structural linguistic metrics;
- automatically determining a quality of the received document based on the comparing, wherein the determining includes computing a quality score corresponding to the determined quality;
- and
- in response to the quality score meeting a quality threshold, ingesting the document into a corpus that is utilized by a question-answering (QA) system.
An information handling system comprising:
- one or more processors;
- a memory coupled to at least one of the processors;
- and
- a set of computer program instructions stored in the memory and executed by at least one of the processors in order to perform actions comprising:
- receiving a document and a document type, wherein the document type identifies a document category to which the received document belongs;
- identifying a set of structural linguistic features found in the document, wherein the set of structural linguistic features are associated with the document type;
- retrieving a set of structural linguistic metrics corresponding to the document type;
- comparing the set of structural linguistic features found in the document to the retrieved set of structural linguistic metrics;
- automatically determining a quality of the received document, based on the comparing, wherein the determining includes computing a quality score corresponding to the determined quality;
- and
- in response to the quality score meeting a quality threshold, ingesting the document into a corpus that is utilized by a question-answering (QA) system.
A computer program product stored in a computer readable storage medium, comprising computer program code that, when executed by an information handling system, performs actions comprising:
- receiving a document and a document type, wherein the document type identifies a document category to which the received document belongs;
- identifying a set of structural linguistic features found in the document, wherein the set of structural linguistic features are associated with the document type;
- retrieving a set of structural linguistic metrics corresponding to the document type;
- comparing the set of structural linguistic features found in the document to the retrieved set of structural linguistic metrics;
- automatically determining a quality of the received document based on the comparing, wherein the determining includes computing a quality score corresponding to the determined quality;
- and
- in response to the quality score meeting a quality threshold, ingesting the document into a corpus that is utilized by a question-answering (QA) system, wherein the ingesting is based on the determined quality.
Dependencies are inferred from the claim text.