Method and apparatus for automatic entity disambiguation

A method for entity disambiguation for implementation by one or more data processors, the method comprising:

• using, by at least one of the data processors, cross-document disambiguation to identify an entity across a plurality of context domains, each domain comprising a finite set of context items, the context items comprising standardized names derived in an entity information extraction phase of within-document disambiguation;
• using, by at least one of the data processors, a logarithm of an inverse name frequency which comprises a number of standard names with which a context item appears as a weight indicating salience of each context item, wherein co-occurrence with a common name provides less indication that two mentions correspond to a same entity than co-occurrence with an uncommon name;
• using, by at least one of the data processors, a sparse count vector for recording all of the items that co-occur with a particular entity;
• creating, by at least one of the data processors, a sparse count vector of title tokens that occur with an entity;
• and
• computing, by at least one of the data processors, inverse name frequency weights for said title tokens.

An article comprising a tangible machine-readable storage medium embodying instructions that when performed by one or more machines result in operations comprising:

• carrying out a within-document co-reference resolution for a plurality of documents;
• aggregating attributes about each entity mentioned in each document;
• and
• using the entity attributes as features in determining which documents concern a same entity;
• the within document co-reference resolution comprising:
• initializing a list of entities as an empty list;
• processing names in order of longest to shortest within each entity type class;
• and
• comparing each name to all entities with which it may be compatible, based on a token hash or database, the comparing being carried out via entity type-specific distance measures.

An article comprising a tangible machine-readable storage medium embodying instructions that when performed by one or more machines result in operations comprising:

• identifying a first plurality of entities by carrying out a within-document co-reference resolution for a first document;
• identifying a second plurality of entities by carrying out a within-document co-reference resolution for a second document;
• identifying at least one common entity that exists in the first plurality of entities and the second plurality of entities;
• and
• verifying the at least one common entity by comparing the first plurality of entities to the second plurality of entities;
• the within document co-reference resolution comprising:
• initializing a list of entities as an empty list;
• processing names in order of longest to shortest within each entity type class;
• and
• comparing each name to all entities with which it may be compatible, based on a token hash or database, the comparing being carried out via entity type-specific distance measures.

An article comprising a tangible machine-readable storage medium embodying instructions that when performed by one or more machines result in operations comprising:

• identifying a first plurality of entities by carrying out a within-document co-reference resolution for a first document;
• identifying a second plurality of entities by carrying out a within-document co-reference resolution for a second document;
• identifying at least one common entity that exists in the first plurality of entities and the second plurality of entities;
• and
• verifying the at least one common entity based on a first context of the at least one common entity in the first plurality of entities and the second context of the at least one common entity in the second plurality of entities;
• the within document co-reference resolution comprising:
• initializing a list of entities as an empty list;
• processing names in order of longest to shortest within each entity type class;
• and
• comparing each name to all entities with which it may be compatible, based on a token hash or database, the comparing being carried out via entity type-specific distance measures.

An article comprising a tangible machine-readable storage medium embodying instructions that when performed by one or more machines result in operations comprising:

• using cross-document disambiguation to identify an entity across a plurality of context domains, each domain comprising a finite set of context items, the context items comprising standardized names derived in an entity information extraction phase of within-document disambiguation;
• using a logarithm of an inverse name frequency which comprises a number of standard names with which a context item appears as a weight indicating salience of each context item, wherein co-occurrence with a common name provides less indication that two mentions correspond to a same entity than co-occurrence with an uncommon name;
• using a sparse count vector for recording all of the items that co-occur with a particular entity;
• creating a sparse count vector of title tokens that occur with an entity;
• and
• computing inverse name frequency weights for said title tokens.