Method, System and Apparatus for Automatic Keyword Extraction
AssigneeUniversity of North Texas
InventorsAndras Csomai, Rada Mihalcea
Life of this patent
Estimating the term…
Abstract
The present invention provides a method and a system for automatic keyword extraction based on supervised or unsupervised machine learning techniques. Novel linguistically-motivated machine learning features are introduced, including discourse comprehension features based on construction integration theory, numeric features making use of syntactic part-of-speech patterns, and probabilistic features based on analysis of online encyclopedia annotations. The improved keyword extraction methods are combined with word sense disambiguation into a system for automatically generating annotations to enrich text with links to encyclopedic knowledge.
Claims35
A method for automatically generating one or more keywords from an electronic document, the method comprising the steps of:
- identifying candidate entries for the keywords;
- constructing a feature vector for each candidate entry, wherein the feature vector comprises at least one feature selected from the group consisting of one or more discourse comprehension features, one or more part-of-speech pattern features, and one or more encyclopedic annotation features;
- assigning a numeric score to each candidate entry based on the feature vector for that candidate entry;
- and
- selecting a specified number of candidate entries to be retained as the keywords.
A computer program embodied on a computer readable medium for automatically generating one or more keywords from an electronic document comprising:
- a code segment for identifying candidate entries for the keywords;
- a code segment for constructing a feature vector for each candidate entry, wherein the feature vector comprises at least one feature selected from the group consisting of one or more discourse comprehension features, one or more part-of-speech pattern features, and one or more encyclopedic annotation features;
- a code segment for assigning a numeric score to each candidate entry based on the feature vector for that candidate entry;
- and
- a code segment for selecting a specified number of entries to be retained as the keywords.
An apparatus for automatically generating one or more keywords from an electronic document, the apparatus comprising;
- an interface to a document storage repository;
- a memory;
- and
- a processor communicably connected to the interface and the memory, wherein the processor (a) identifies candidate entries for the keywords, (b) constructs a feature vector for each candidate entry, wherein the feature vector comprises at least one feature selected from the group consisting of one or more discourse comprehension features, one or more part-of-speech pattern features, and one or more encyclopedic annotation features, (c) assigns a numeric score to each candidate entry based on the feature vector for that candidate entry, and (d) selects a specified number of entries to be retained as the keywords.
A system for automatically generating one or more keywords from an electronic document, the system comprising:
- a network;
- one or more client computers communicably connected to the network;
- one or more server computers communicably connected to the network;
- one or more document storage repositories communicably connected to the network, to one or more of the client computers, or to one or more of the server computers;
- and
- a processor within at least one of the client computers or server computers, wherein the processor (a) identifies candidate entries for the keywords, (b) constructs a feature vector for each candidate entry, wherein the feature vector comprises at least one feature selected from the group consisting of one or more discourse comprehension features, one or more part-of-speech pattern features, and one or more encyclopedic annotation features, (c) assigns a numeric score to each candidate entry based on the feature vector for that candidate entry, and (d) selects a specified number of entries to be retained as the keywords.
Dependencies are inferred from the claim text.