# US 2015/0106079 A1 · Ontology-driven annotation confidence levels for natural language processing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2015/0106079 A1
- Document ID: US20150106079A1
- Status: Published application
- Kind: A1
- Application number: 14/055,185
- Filed: 2013-10-16
- Published: 2015-04-16
- Earliest claimed: 2013-10-16
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: James E. Bostick; John M. Ganci, JR.; Ibrahim Salah; Craig M. Trim
- Web page: https://patentgrep.com/patent/US20150106079A1

## CPC

- G06F 40/289: Phrasal analysis, e.g. finite state techniques or chunking
- G06F 16/3334: {Selection or weighting of terms from queries, including natural language queries}
- G06F 40/205: Parsing
- G06F 40/211: Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
- G06F 40/253: Grammatical analysis; Style critique
- G06F 40/30: Semantic analysis
- G06F 40/40: Processing or translation of natural language (natural language analysis G06F40/20; semantic analysis G06F40/30)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

An approach for determining a combination of terms that represents subject matter of a natural language sentence is provided. Numbers of words from a beginning of the sentence to terms in the sentence that match terms in the combination of terms are determined. The sentence is divided into natural language phrases including a complex phrase and first and second simple phrases extracted from the complex phrase. Based in part on (a) the numbers of words from the beginning of the sentence to the terms in the sentence that match terms in the combination of terms, (b) whether all terms of the combination are contained in the first and/or second simple phrases, and (c) whether all terms of the combination are contained in the complex phrase but not contained in the first and/or second simple phrases, how well the combination of terms represents the subject matter is determined.

## Claims

### Claim 1

A method of determining a combination of terms that represents subject matter of a natural language sentence, the method comprising the steps of:
a computer determining respective numbers of words from a beginning of the sentence to respective terms in the sentence that match terms in the combination of terms;
the computer dividing the sentence in a multiplicity of natural language phrases including a complex phrase and first and second simple phrases extracted from the complex phrase, the complex phrase being less than an entirety of the sentence; and
based in part on (a) the respective numbers of words from the beginning of the sentence to respective terms in the sentence that match terms in the combination of terms, (b) whether all terms of the combination are contained in the first and/or second simple phrases, and (c) whether all terms of the combination are contained in the complex phrase but not contained in the first and/or second simple phrases, the computer determining how well the combination of terms represents the subject matter of the sentence.

### Claim 2 (depends on 1)

The method of claim 1, further comprising the step of:
the computer determining whether a negation is included in a phrase in the sentence, the phrase including a term included in the terms of the sentence that match the terms in the combination of terms, wherein the step of determining how well the combination of terms represents the subject matter of the sentence is based in part on the negation being included in the phrase in the sentence.

### Claim 3 (depends on 1)

The method of claim 1, wherein the step of determining the respective numbers of words includes;
determining a first number of words from the beginning of the sentence to a first term in the sentence that matches a first term in the combination of terms;
determining a second number of words from the beginning of the sentence to a second term in the sentence that matches a second term in the combination of terms; and
determining a difference between the first and second numbers of words, wherein the step of determining how well the combination of terms represents the subject matter of the sentence is further based in part on the difference between the first and second numbers of words.

### Claim 4 (depends on 3)

The method of claim 3, further comprising the steps of:
the computer determining whether the difference between the first and second numbers of words exceeds a threshold;
in response to determining the difference between the first and second numbers of words exceeds the threshold, the computer determining a first amount by which the difference exceeds the threshold;
the computer determining a second amount by multiplying the first amount by a factor; and
the computer adjusting a confidence level indicating how well the combination of terms represents the subject matter of the sentence by subtracting the second amount from the confidence level.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
the computer determining a confidence level exceeds a threshold, the confidence level indicating how well the combination of terms represents the subject matter of the sentence;
based on the confidence level exceeding the threshold, the computer retrieving from a knowledge base contextual information related to the subject matter of the sentence; and
based on the confidence level and the retrieved contextual information, the computer making an inference about the subject matter of the sentence.

### Claim 6 (depends on 1)

The method of claim 1, further comprising the steps of:
the computer receiving an ontology that includes rules; and
the computer forming the combination of terms based on the rules included in the ontology.

### Claim 7 (depends on 1)

The method of claim 1, further comprising the steps of:
the computer generating a table having a top row and other rows, the top row including entries that include respective words in the sentence that match terms in the combination, the other rows including entries that include the multiplicity of natural language phrases, the other rows including first and second rows, the first row including the first and second simple phrases, and the second row including the complex phrase; and
the computer determining respective numbers of rows from the words in the top row that match the terms in the combination to the first row if all terms in the combination are contained in the first and/or second simple phrases, or to the second row if all terms in the combination are contained in the complex phrase but not contained in the first and/or second simple phrases,
wherein the step of determining how well the combination of terms represents the subject matter of the sentence is further based in part on the numbers of rows from the words in the top row to the first or second row.

### Claim 8 (depends on 7)

The method of claim 7, wherein the step of determining the respective numbers of rows includes:
the computer determining a difference between first and second numbers of rows included in the numbers of rows;
the computer determining an amount by multiplying the difference by a factor; and
the computer adjusting a confidence level by subtracting the amount from the confidence level, the confidence level indicating how well the combination of terms represents the subject matter of the sentence.

### Claim 9

A computer program product for determining a combination of terms that represents subject matter of a natural language sentence, the computer program product comprising:
one or more computer-readable storage devices and program instructions stored on the one or more storage devices, the program instructions comprising:
program instructions to determine respective numbers of words from a beginning of the sentence to respective terms in the sentence that match terms in the combination of terms;
program instructions to divide the sentence in a multiplicity of natural language phrases including a complex phrase and first and second simple phrases extracted from the complex phrase, the complex phrase being less than an entirety of the sentence; and
program instructions to determine, based in part on (a) the respective numbers of words from the beginning of the sentence to respective terms in the sentence that match terms in the combination of terms, (b) whether all terms of the combination are contained in the first and/or second simple phrases, and (c) whether all terms of the combination are contained in the complex phrase but not contained in the first and/or second simple phrases, how well the combination of terms represents the subject matter of the sentence.

### Claim 10 (depends on 9)

The computer program product of claim 9, further comprising:
program instructions, stored on at least one of the one or more storage devices, to determine whether a negation is included in a phrase in the sentence, the phrase including a term included in the terms of the sentence that match the terms in the combination of terms, wherein the program instructions to determine how well the combination of terms represents the subject matter of the sentence includes program instructions to determine, based in part on the negation being included in the phrase of the sentence, how well the combination of terms represents the subject matter of the sentence.

### Claim 11 (depends on 9)

The computer program product of claim 9, wherein the program instructions to determine the respective numbers of words includes:
program instructions to determine a first number of words from the beginning of the sentence to a first term in the sentence that matches a first term in the combination of terms;
program instructions to determine a second number of words from the beginning of the sentence to a second term in the sentence that matches a second term in the combination of terms; and
program instructions to determine a difference between the first and second numbers of words, wherein the program instructions to determine how well the combination of terms represents the subject matter of the sentence includes program instructions to determine, based in part on the difference between the first and second numbers of words, how well the combination of terms represents the subject matter in the sentence.

### Claim 12 (depends on 11)

The computer program product of claim 11, further comprising:
program instructions, stored on at least one of the one or more storage devices, to determine whether the difference between the first and second numbers of words exceeds a threshold;
program instructions, stored on at least one of the one or more storage devices, to determine, in response to determining the difference between the first and second numbers of words exceeds the threshold, a first amount by which the difference exceeds the threshold;
program instructions, stored on at least one of the one or more storage devices, to determine a second amount by multiplying the first amount by a factor; and
program instructions, stored on at least one of the one or more storage devices, to adjust a confidence level by subtracting the second amount from the confidence level, the confidence level indicating how well the combination of terms represents the subject matter of the sentence.

### Claim 13 (depends on 9)

The computer program product of claim 9, further comprising:
program instructions, stored on at least one of the one or more storage devices, to determine a confidence level exceeds a threshold, the confidence level indicating how well the combination of terms represents the subject matter of the sentence;
program instructions, stored on at least one of the one or more storage devices, to retrieve from a knowledge base, based on the confidence level exceeding the threshold, contextual information related to the subject matter of the sentence; and
program instructions, stored on at least one of the one or more storage devices, to make, based on the confidence level and the retrieved contextual information, an inference about the subject matter of the sentence.

### Claim 14 (depends on 9)

The computer program product of claim 9, further comprising:
program instructions, stored on at least one of the one or more storage devices, to receive an ontology that includes rules; and
program instructions, stored on at least one of the one or more storage devices, to form the combination of terms based on the rules included in the ontology.

### Claim 15

A computer system for determining a combination of terms that represents subject matter of a natural language sentence, the computer system comprising:
one or more processors;
one or more computer-readable memories;
one or more computer-readable storage devices; and
program instructions stored on the one or more storage devices for execution by the one or more processors via the one or more memories, the program instructions comprising:
first program instructions to determine respective numbers of words from a beginning of the sentence to terms in the sentence that match terms in the combination of terms;
second program instructions to divide the sentence in a multiplicity of natural language phrases including a complex phrase and first and second simple phrases extracted from the complex phrase, the complex phrase being less than an entirety of the sentence; and
third program instructions to determine, based in part on (a) the respective numbers of words from the beginning of the sentence to respective terms in the sentence that match terms in the combination of terms, (b) whether all terms of the combination are contained in the first and/or second simple phrases, and (c) whether all terms of the combination are contained in the complex phrase but not contained in the first and/or second simple phrases, how well the combination of terms represents the subject matter of the sentence.

### Claim 16 (depends on 15)

The computer system of claim 15, further comprising:
fourth program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to determine whether a negation is included in a phrase in the sentence, the phrase including a term included in the terms of the sentence that match the terms in the combination of terms, wherein the third program instructions to determine how well the combination of terms represents the subject matter of the sentence includes fifth program instructions to determine, based in part on the negation being included in the phrase of the sentence, how well the combination of terms represents the subject matter of the sentence.

### Claim 17 (depends on 15)

The computer system of claim 15, wherein the first program instructions to determine the respective numbers of words includes:
fourth program instructions to determine a first number of words from the beginning of the sentence to a first term in the sentence that matches a first term in the combination of terms;
fifth program instructions to determine a second number of words from the beginning of the sentence to a second term in the sentence that matches a second term in the combination of terms; and
sixth program instructions to determine a difference between the first and second numbers of words, wherein the third program instructions to determine how well the combination of terms represents the subject matter of the sentence includes seventh program instructions to determine, based in part on the difference between the first and second numbers of words, how well the combination of terms represents the subject matter of the sentence.

### Claim 18 (depends on 17)

The computer system of claim 17, further comprising:
eighth program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to determine whether the difference between the first and second numbers of words exceeds a threshold;
ninth program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to determine, in response to determining the difference between the first and second numbers of words exceeds the threshold by an execution of the eighth program instructions, a first amount by which the difference exceeds the threshold;
tenth program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to determine a second amount by multiplying the first amount by a factor; and
eleventh program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to adjust a confidence level by subtracting the second amount from the confidence level, the confidence level indicating how well the combination of terms represents the subject matter of the sentence.

### Claim 19 (depends on 15)

The computer system of claim 15, further comprising:
fourth program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to determine a confidence level exceeds a threshold, the confidence level indicating how well the combination of terms represents the subject matter of the sentence;
fifth program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to retrieve from a knowledge base, based on the confidence level exceeding the threshold, contextual information related to the subject matter of the sentence; and
sixth program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to make, based on the confidence level and the retrieved contextual information, an inference about the subject matter of the sentence.

### Claim 20 (depends on 15)

The computer system of claim 15, further comprising:
fourth program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to receive an ontology that includes rules; and
program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to form the combination of terms based on the rules included in the ontology.
