Readability awareness in natural language processing systems
AssigneeInternational Business Machines Corporation
InventorsDonna K. Byron, Devendra Goyal, Lakshminarayanan Krishnamurthy, Priscilla Santos Moraes, Michael C. Smith
Life of this patent
Estimating the term…
Abstract
Electronic natural language processing in a natural language processing (NLP) system, such as a Question-Answering (QA) system. A receives electronic text input, in question form, and determines a readability level indicator in the question. The readability level indicator includes at least a grammatical error, a slang term, and a misspelling type. The computer determines a readability level for the electronic text input based on the readability level indicator, and retrieves candidate answers based on the readability level.
Claims17
A method for electronic natural language processing in an electronic natural language processing (NLP) system comprising a question-answering (QA) pipeline having a plurality of processing stages, wherein one or more of steps of the method are performed by one or more of the plurality of processing stages, comprising:
- receiving, by the QA pipeline, a plurality of natural language documents;
- determining readability level indicators in the plurality of natural language documents;
- filtering one or more of the natural language documents, by the QA pipeline, to exclude one or more natural language documents from processing by at least one other processing stage;
- and
- providing, by the QA pipeline in response to receiving a query text, at least one natural language document whose readability level is within a threshold distance of a readability level of the query text, wherein the readability level of the query text is based on one or more readability level indicators including at least one of a grammatical error, a slang term, and a misspelling type in the query text.
A computer system for electronic natural language processing (NLP), comprising:
- one or more computer devices each having one or more processors and one or more tangible storage devices;
- and
- a program embodied on at least one of the one or more storage devices, the program having a plurality of program instructions for execution by the one or more processors, the program instructions comprising programming instructions for a question-answering (QA) pipeline having a plurality of processing stages, wherein one or more of steps of a method are performed by executing one or more of the plurality of processing stages, and the computer programming instructions further comprising instructions for:
- receiving, by the QA pipeline, a plurality of natural language documents;
- determining readability level indicators in the plurality of natural language documents;
- filtering one or more of the natural language documents, by the QA pipeline, to exclude one or more natural language documents from processing by at least one other processing stage;
- and
- providing, by the QA pipeline in response to receiving a query text, at least one natural language document whose readability level is within a threshold distance of a readability level of the query text, wherein the readability level of the query text is based on one or more readability level indicators including at least one of a grammatical error, a slang term, and a misspelling type in the query text.
A computer program product for electronic natural language processing (NLP) in an electronic NLP system comprising a question-answering (QA) pipeline having a plurality of processing stages, the computer program product comprising a non-transitory tangible storage device having program code embodied therewith, the program code executable by a processor of a computer to perform a method, the method comprising:
- receiving, by the computer executing one or more processing stages of the QA pipeline, a plurality of natural language documents;
- determining, by the computer executing one or more processing stages of the QA pipeline, readability level indicators in the plurality of natural language documents;
- filtering, by the computer executing one or more processing stages of the QA pipeline, one or more of the natural language documents, by the QA pipeline, to exclude one or more natural language documents from processing by at least one other processing stage;
- and
- providing, by the computer executing one or more processing stages of the QA pipeline, in response to receiving a query text, at least one natural language document whose readability level is within a threshold distance of a readability level of the query text, wherein the readability level of the query text is based on one or more readability level indicators including at least one of a grammatical error, a slang term, and a misspelling type in the query text.
Dependencies are inferred from the claim text.