# US 2019/0095526 A1 · Keyphrase extraction system and method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0095526 A1
- Document ID: US20190095526A1
- Status: Published application
- Kind: A1
- Application number: 15/832,892
- Filed: 2017-12-06
- Published: 2019-03-28
- Earliest claimed: 2017-09-22
- Assignees: Druva Technologies Pte. Ltd.
- Applicants: Druva Technologies PTE. LTD.
- Inventors: Bhave Adwait
- Web page: https://patentgrep.com/patent/US20190095526A1

## CPC

- G06F 16/3344: {using natural language analysis}
- G06F 16/345: {Summarisation for human users}
- G06F 40/268: Morphological analysis
- G06F 40/289: Phrasal analysis, e.g. finite state techniques or chunking
- G06F 40/30: Semantic analysis
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A keyphrase extraction system and method is provided. The keyphrase extraction system includes a memory having computer-readable instructions stored therein. The keyphrase extraction system also includes a processor configured to access a document. The processor is configured to identify a plurality of candidate phrases from the document based upon a part-of-speech tag pattern. Each of the plurality of candidate phrases comprises one or more candidate terms. In addition, the processor is further configured to access an external knowledge base to determine a vocabulary frequency count of the one or more candidate terms. The vocabulary frequency count of the one or more candidate terms corresponds to a count of appearance of the respective candidate term in a plurality of documents accessible by the external knowledge base. Further, the processor is configured to estimate a phrase score for each of the plurality of candidate phrases based upon the vocabulary frequency count of the one or more candidate terms of each of the plurality of candidate phrases. Furthermore, the processor is configured to filter the plurality of candidate phrases based upon the estimated phrase score and pre-determined thresholds to determine one or more key phrases present in the document.

## Claims

### Claim 1

A keyphrase extraction system comprising:
a memory having computer-readable instructions stored therein; and
a processor configured to:
access a document;
identify a plurality of candidate phrases from the document based upon a part-of-speech tag pattern, wherein each of the plurality of candidate phrases comprises one or more candidate terms;
access an external knowledge base to determine a vocabulary frequency count of the one or more candidate terms, wherein the vocabulary frequency count of the one or more candidate terms corresponds to a count of appearance of the respective candidate term in a plurality of documents accessible by the external knowledge base;
estimate a phrase score for each of the plurality of candidate phrases based upon the vocabulary frequency count of the one or more candidate terms of each of the plurality of candidate phrases; and
filter the plurality of candidate phrases based upon the estimated phrase score and pre-determined thresholds to determine one or more key phrases present in the document.

### Claim 2 (depends on 1)

The system of claim 1, wherein the processor is further configured to execute the computer-readable instructions to preprocess the document to remove noise elements related to bad encoding, operating system specific formatting, file conversion defects, or combinations thereof.

### Claim 3 (depends on 1)

The system of claim 1, wherein the processor is further configured to execute the computer-readable instructions to identify a plurality of candidate phrases from the document using a noun phrase extraction technique.

### Claim 4 (depends on 1)

The system of claim 1, wherein the processor is further configured to execute the computer-readable instructions to:
filter the plurality of candidate phrases based upon pre-determined filters to select a subset of the candidate phrases; and
access the external knowledge base to determine a vocabulary frequency count of the one or more candidate terms of the selected candidate phrases.

### Claim 5 (depends on 4)

The system of claim 4, wherein the processor is further configured to execute the computer-readable instructions to remove the candidate phrases that comprise special characters, terms from a stop word list, terms with less than 3 alphanumeric characters, more than 4 terms, or combinations thereof.

### Claim 6 (depends on 4)

The system of claim 4, wherein the processor is further configured to execute the computer-readable instructions to access Google books corpus to determine the vocabulary frequency count of the one or more candidate terms of the selected candidate phrases, wherein the vocabulary frequency count of the candidate terms is indicative of rarity of the respective candidate term.

### Claim 7 (depends on 1)

The system of claim 1, wherein the processor is further configured to execute the computer-readable instructions to:
estimate a term probability of each of the one or more candidate terms of the selected candidate phrases;
estimate a word score for each of the candidate terms based upon the estimated term probabilities;
estimate a phrase score of each of the selected candidate phrases based upon the word score; and
estimate a key phrase importance score of the selected candidate phrases based upon the phrase score.

### Claim 8 (depends on 7)

The system of claim 7, wherein the processor is further configured to execute the computer-readable instructions to estimate the term probability of the candidate terms in accordance with the relationship:
term


probability
=
vocabulary


frequency


count
sum

(
all


vocabulary


frequency


count
)
where: vocabulary frequency count is word count of appearance of the respective candidate term; and
sum (all vocabulary frequency count) is sum of word counts of all the terms.

### Claim 9 (depends on 8)

The system of claim 8, wherein the processor is further configured to execute the computer-readable instructions to estimate the word score of the candidate terms based upon a product of term probabilities of all the candidate terms.

### Claim 10 (depends on 9)

The system of claim 9, wherein the processor is further configured to execute the computer-readable instructions to estimate the phrase score of the candidate phrases in accordance with the relationship:
phrase score=(word scorê(log1p(vocabulary frequency count)))*(word count̂5)

### Claim 11 (depends on 1)

The system of claim 1, wherein the processor is further configured to execute the computer-readable instructions to determine a ranking of each of the candidate phrases based upon the estimated phrase score.

### Claim 12

A method for extracting key phrases present in a document, the method comprising:
identifying a plurality of candidate phrases from the document based upon a part-of-speech tag pattern, wherein each of the plurality of candidate phrases comprises one or more candidate terms;
determining a vocabulary frequency count of the one or more candidate terms of each of the plurality of candidate phrases using an external knowledge base;
estimating a phrase score for each of the plurality of candidate phrases based on the vocabulary frequency count of the one or more candidate terms of the respective candidate phrases; and
determining one or more key phrases present in the document based on the estimated phrase score of each of the plurality of candidate phrases.

### Claim 13 (depends on 12)

The method of claim 12, further comprising identifying the plurality of candidate phrases from the document based upon a noun phrase extraction technique.

### Claim 14 (depends on 12)

The method of claim 12, further comprising:
preprocessing the document to remove noise elements to generate a pre-processed document prior to identifying of the plurality of candidate phrases;
filtering the plurality of candidate phrases to generate a subset of the candidate phrases; and
determining the vocabulary frequency count of the one or more candidate terms of each of the subset of candidate phrases using the external knowledge base.

### Claim 15

A keyphrase extraction system comprising:
a memory having computer-readable instructions stored therein; and
a processor configured to extract one or more key phrases present in a document, wherein the processor comprises:
a candidate phrase extractor configured to identify a plurality of candidate phrases from the document using a noun phrase extraction technique, wherein each of the plurality of candidate phrases comprises one or more candidate terms;
a word count determination module configured to access an external knowledge base to determine a word count of the one or more candidate terms, wherein the word count of the one or more candidate terms corresponds to a count of appearance of the respective candidate term in a plurality of documents accessible by the external knowledge base;
a key phrase selection module configured to estimate a phrase score for each of the plurality of candidate phrases based upon the word count of the one or more candidate terms of each of the plurality of candidate phrases and to determine one or more key phrases based upon the estimated phrase score.

### Claim 16 (depends on 15)

The system of claim 15, wherein the key phrase selection module is further configured to:
estimate a term probability of each of the candidate terms of the identified candidate phrases;
estimate a word score for each of the candidate terms based upon the estimated term probabilities; and
estimate the phrase score of each of the selected candidate phrases based upon the word score.

### Claim 17 (depends on 15)

The system of claim 15, wherein the word count determination module is configured to access Google books corpus to determine the word count of the one or more candidate terms

### Claim 18 (depends on 15)

The system of claim 15, wherein the key phrase selection module is further configured to apply threshold filters to select the one or more key phrases present in the document.

### Claim 19 (depends on 15)

The system of claim 15, wherein the system is configured to identify key phrases present in documents across a plurality of domains.

### Claim 20 (depends on 15)

The system of claim 15, wherein the key phrase selection module is further configured to rank the identified key phrases based upon the estimated phrase score of each of the selected key phrases.
