US 8,370,328 B2Granted

System and method for creating and maintaining a database of disambiguated entity mentions and relations from a corpus of electronic documents

AssigneeCOMSORT, Inc.

InventorsMichael A. Woytowitz, Marshall Wells Hawks

Life of this patent

Estimating the term…

Loading patent…

Abstract

Method and apparatus for creating an electronic database of disambiguated entity mentions and relations from a corpus of electronic documents. The invention automatically extracts from the corpus of electronic documents mentions about entities (e.g., references to people, organizations or places), parses the entity mentions into “mention objects,” and executes a series of grouping, comparison and hierarchical fuzzy object clustering algorithms to cluster together in an electronic database all of the mention objects referring to the same entity and all of the mention objects (e.g. “people”) associated with each other by a relationship (e.g., “co-authors” or “family members”). The resulting electronic database of disambiguated entity mentions and relations, which may comprise, for example, an XML document, a relational database or hierarchical database, is structured to permit useful recordation, access, review and display of all of the mentions and relations associated with a particular entity or collection of entities.

Claims52

Dependent claims

A method for creating an electronic database of disambiguated entity mentions from a corpus of electronic documents using a microprocessor, the method comprising:

(a) executing on the microprocessor a data harvesting module to automatically extract entity mentions from the electronic documents in the corpus and parse the entity mentions into mention objects;
(b) executing on the microprocessor a mention group creation module to create one or more mention groups by automatically grouping the mention objects together according to a distinguishing attribute common to a given class of mention objects;
(c) selecting a mention group from the one or more mention groups for comparison processing;
(d) executing on the microprocessor a collection of comparison modules that automatically (i) compares every mention object in the selected mention group with every other mention object in the selected mention group to produce a collection of comparison algorithm scores for every pair of mention objects in the selected mention group, and (ii) generates an overall confidence score for every pair of mention objects in the selected mention group based on the collection of comparison algorithm scores for said every pair of mention objects;
(e) executing on the microprocessor an entity object creation module to create one or more new entity objects for the selected mention group by automatically (i) grouping together mention objects with other mention objects, based on the confidence scores of each pair of mention objects and a specified confidence threshold, wherein pairs of mention objects having a confidence score greater than or equal to the specified threshold are assigned to the same new entity object, and (ii) merging previously-created entity objects with other previously-created entity objects, based on the confidence scores of each pair of entity objects, and a specified confidence threshold, wherein pairs of entity objects having a confidence score greater than or equal to the specified threshold are assigned to the same new entity object;
(f) storing said one or more new entity objects in the electronic database of disambiguated entity mentions;
and
(g) repeating steps (c) through (f) above until all of the one or more mention groups have been comparison processed.

5 more elements

An apparatus for creating an electronic database of disambiguated entity mentions from a corpus of electronic documents, comprising:

a microprocessor;
a data harvesting module comprising program instructions that, when executed by microprocessor, will cause the microprocessor to automatically extract entity mentions from the corpus of electronic documents and parse the entity mentions to produce one or more mention objects;
a mention group creation module comprising program instructions that, when executed by microprocessor, will cause the microprocessor to automatically create one or more mention groups by automatically grouping mention objects together according to a distinguishing attribute common to a given class of mention objects;
a collection of comparison modules having program instructions that, when executed by microprocessor, will cause the microprocessor to automatically (i) compare every mention object in a selected mention group with every other mention object in the selected mention group to produce a collection of comparison algorithm scores for every pair of mention objects in the selected mention group, and (ii) generate an overall confidence score for every pair of mention objects in the selected mention group based on the collection of comparison algorithm scores for said every pair;
and
an entity object creation module having program instructions that, when executed by microprocessor, will cause the microprocessor to automatically create in the electronic database one or more new entity objects for the selected mention group by automatically
(i) grouping together mention objects with other mention objects, based on the confidence scores of each pair of mention objects and a specified confidence threshold, wherein pairs of mention objects having a confidence score greater than or equal to the specified threshold are assigned to the same new entity object, and
(ii) merging previously-created entity objects with other previously-created entity objects, based on the confidence scores of each pair of entity objects, and a specified confidence threshold, wherein pairs of entity objects having a confidence score greater than or equal to the specified threshold are assigned to the same new entity object.

5 more elements

An apparatus for augmenting an electronic database of disambiguated entity mentions, comprising:

a microprocessor;
a document information database having a set of records that uniquely identify each electronic document in the corpus that was used to create the electronic database of disambiguated mention objects;
a data harvesting module comprising program instructions that cause the microprocessor to automatically (i) read and extract entity mentions from each new electronic document based on the set of records uniquely identifying the used electronic documents, (ii) parse the entity mentions of each new electronic document into a plurality of new mention objects in accordance with the predefined structure, and (iii) store the plurality of new mention objects in a mention object database;
a mention group creation module that retrieves the plurality of new mention objects from the mention object database and groups them according to a distinguishing attribute common to a given class of mention objects, by first determining whether the each mention object should be associated with an existing mention group or a new mention group;
a set of comparison rules;
a collection of comparison algorithms that compares each new mention object in a selected mention group with every other new mention object in the selected mention group to produce a collection of algorithm scores for each comparison pair, and then produces a confidence score for each comparison pair based on the collection of algorithm scores for that comparison pair;
and
an entity object creation module having program instructions that cause the microprocessor to automatically create and store in the electronic database of disambiguated entity mentions one or more new entity objects for the selected mention group by automatically
(i) grouping together mention objects with other mention objects, based on the confidence scores of each pair of mention objects and a specified confidence threshold, wherein pairs of mention objects having a confidence score greater than or equal to the specified threshold are assigned to the same new entity object, and
(ii) merging previously-created entity objects with other previously-created entity objects, based on the confidence scores of each pair of entity objects, and a specified confidence threshold, wherein pairs of entity objects having a confidence score greater than or equal to the specified threshold are assigned to the same new entity object.

7 more elements

A non-transitory computer-readable storage medium with an executable program for creating an electronic database of disambiguated entity mentions from a corpus of electronic documents stored thereon, wherein the executable program comprises instructions to cause a microprocessor to:

(a) automatically extract entity mentions from the corpus of electronic documents and parse the entity mentions into mention objects;
(b) create one or more mention groups by automatically grouping the mention objects together according to a distinguishing attribute common to a given class of mention objects;
(c) select a mention group from the one or more mention groups for comparison processing;
(d) automatically compare every mention object in the selected mention group with every other mention object in the selected mention group to produce a collection of comparison algorithm scores for every pair of mention objects in the selected mention group;
(e) generate an overall confidence score for every pair of mention objects in the selected mention group based on the collection of comparison algorithm scores for said every pair;
(f) create in the electronic database of disambiguated entity mentions one or more new entity objects for the selected mention group by automatically (i) grouping together mention objects with other mention objects, based on the confidence scores of each pair of mention objects and a specified confidence threshold, wherein pairs of mention objects having a confidence score greater than or equal to the specified threshold are assigned to the same new entity object, and (ii) merging previously-created entity objects with other previously-created entity objects, based on the confidence scores of each pair of entity objects, and a specified confidence threshold, wherein pairs of entity objects having a confidence score greater than or equal to the specified threshold are assigned to the same new entity object;
(g) repeat steps (c) through (f) above until all of the one or more mention groups have been comparison processed.

4 more elements

Dependencies are inferred from the claim text.