Back to results
US 11,907,858 B2Granted
Entity disambiguation
AssigneeYahoo Assets LLC
InventorsAasish Pappu, Roi Blanco, Yashar Mehdad, Amanda Stent, Kapil Thadani
Life of this patent
Estimating the term…
Abstract
One or more computing devices, systems, and/or methods for entity disambiguation are provided. For example, a document may be analyzed to identify a first mention and a second mention. One or more techniques may be used to select and link a candidate entity, from a first set of candidate entities, to the first mention and select and link a candidate entity, from a second set of candidate entities, to the second mention.
Claims19
A computer-implemented method for linking a mention to an entity in one or more documents hosted on at least one of a document host or a document portal, comprising:
- receiving a request to link mentions in a document to entities, wherein the document is an article published by a source;
- retrieving, by a server, the document from the source via a network connection;
- processing the retrieved document by:
- scanning the document, using at least one of natural language processing or word/phrase matching, to identify a plurality of mentions, in the document, comprising a first mention in the document and a second mention in the document;
- searching a knowledge base for candidate entities that match the first mention identified in the document and retrieving, from the knowledge base, a first set of candidate entities, comprising a first candidate entity and a second candidate entity, corresponding to the first mention;
- and
- searching the knowledge base for candidate entities that match the second mention identified in the document and retrieving, from the knowledge base, a second set of candidate entities, comprising a third candidate entity and a fourth candidate entity, corresponding to the second mention;
- processing a plurality of documents extracted from the knowledge base to generate processed documents by transforming one or more hyperlinks in each document to a canonical form for an associated entity;
- training an entity embedding model using the processed documents, wherein a first layer of the entity embedding model is associated with modeling context of entities of the knowledge base and a second layer of the entity embedding model is associated with modeling context of text spans of the knowledge base;
- generating, using the entity embedding model and based upon a forward pass of the plurality of mentions, a first matrix indicative of a first likelihood of the first mention corresponding to the first candidate entity, a second likelihood of the first mention corresponding to the second candidate entity, a third likelihood of the second mention corresponding to the third candidate entity, and a fourth likelihood of the second mention corresponding to the fourth candidate entity;
- generating, using the entity embedding model and based upon a backward pass of the plurality of mentions, a second matrix indicative of a fifth likelihood of the first mention corresponding to the first candidate entity, a sixth likelihood of the first mention corresponding to the second candidate entity, a seventh likelihood of the second mention corresponding to the third candidate entity, and an eighth likelihood of the second mention corresponding to the fourth candidate entity;
- selecting, from the first set of candidate entities, the first candidate entity but not the second candidate entity and linking the first candidate entity but not the second candidate entity to the first mention based upon the first matrix and the second matrix;
- selecting, from the second set of candidate entities, the third candidate entity but not the fourth candidate entity and linking the third candidate entity but not the fourth candidate entity to the second mention based upon the first matrix and the second matrix, wherein at least one of the selecting the first candidate entity or the selecting the third candidate entity is based upon at least one of:
- local context corresponding to one or more verbs used in the article being indicative of actions associated with a first profession associated with at least one of the first candidate entity or the third candidate entity, wherein one or more second verbs not used in the article are indicative of actions associated with a second profession associated with at least one of the second candidate entity or the fourth candidate entity;
- or
- locational context determined based upon GPS of a device of a user;
- determining a topic of the article based upon the linking of the first candidate entity but not the second candidate entity to the first mention and the linking of the third candidate entity but not the fourth candidate entity to the second mention;
- receiving a second request to link second mentions in a second document to second entities, wherein the second document is a second article published by a second source;
- retrieving, by the server, the second document from the source via a second network connection;
- processing the retrieved second document by:
- scanning the second document, using at least one of natural language processing or word/phrase matching, to identify a second plurality of mentions, in the second document, comprising a third mention and a fourth mention;
- searching the knowledge base for candidate entities that match the third mention identified in the second document and retrieving, from the knowledge base, a third set of candidate entities, comprising a fifth candidate entity and a sixth candidate entity, corresponding to the third mention;
- and
- searching the knowledge base for candidate entities that match the fourth mention identified in the second document and retrieving, from the knowledge base, a fourth set of candidate entities, comprising a seventh candidate entity and an eighth candidate entity, corresponding to the fourth mention;
- generating, using the entity embedding model and based upon a forward pass of the second plurality of mentions, a third matrix indicative of a ninth likelihood of the third mention corresponding to the fifth candidate entity, a tenth likelihood of the third mention corresponding to the sixth candidate entity, an eleventh likelihood of the fourth mention corresponding to the seventh candidate entity, and a twelfth likelihood of the fourth mention corresponding to the eighth candidate entity;
- generating, using the entity embedding model and based upon a backward pass of the second plurality of mentions, a fourth matrix indicative of a thirteenth likelihood of the third mention corresponding to the fifth candidate entity, a fourteenth likelihood of the third mention corresponding to the sixth candidate entity, a fifteenth likelihood of the fourth mention corresponding to the seventh candidate entity, and a sixteenth likelihood of the fourth mention corresponding to the eighth candidate entity;
- selecting, from the third set of candidate entities, the fifth candidate entity but not the sixth candidate entity and linking the fifth candidate entity but not the sixth candidate entity to the third mention based upon the third matrix and the fourth matrix;
- selecting, from the fourth set of candidate entities, the seventh candidate entity but not the eighth candidate entity and linking the seventh candidate entity but not the eighth candidate entity to the fourth mention based upon the third matrix and the fourth matrix, wherein at least one of the selecting the fifth candidate entity or the selecting the seventh candidate entity is based upon at least one of:
- second local context corresponding to one or more third verbs used in the second article being indicative of actions associated with a third profession associated with at least one of the fifth candidate entity or the seventh candidate entity, wherein one or more fourth verbs not used in the second article are indicative of actions associated with a fourth profession associated with at least one of the sixth candidate entity or the eighth candidate entity;
- or
- second locational context determined based upon GPS of a second device of a second user;
- determining a second topic of the second article based upon the linking of the fifth candidate entity but not the sixth candidate entity to the third mention and the linking of the seventh candidate entity but not the eighth candidate entity to the fourth mention;
- and
- organizing and presenting, by the server, the article and the second article among a plurality of articles on at least one of a document host or a document portal based upon the topic of the article and the second topic of the second article.
31 more elements
A computing device comprising:
- a processor;
- and
- memory comprising processor-executable instructions that when executed by the processor cause performance of operations, the operations comprising:
- processing a plurality of documents extracted from a knowledge base to generate processed documents by transforming one or more hyperlinks in each document to a canonical form for an associated entity;
- training an entity embedding model using the processed documents, wherein a first layer of the entity embedding model models context of entities of the knowledge base and a second layer of the entity embedding model models context of text spans of the knowledge base;
- analyzing a document to identify a plurality of mentions, comprising a first mention and a second mention, in the document;
- searching the knowledge base for candidate entities that match the first mention identified in the document and retrieving, from the knowledge base and using the entity embedding model, a first set of candidate entities, comprising a first candidate entity and a second candidate entity, corresponding to the first mention;
- searching the knowledge base for candidate entities that match the second mention identified in an image in the document and retrieving, from the knowledge base and using the entity embedding model, a second set of candidate entities, comprising a third candidate entity and the first candidate entity, corresponding to the second mention;
- determining a first mention location corresponding to the first mention in a vector space;
- determining a second mention location corresponding to the second mention in the vector space;
- determining a first candidate location corresponding to the first candidate entity in the vector space;
- determining a second candidate location corresponding to the second candidate entity in the vector space;
- determining a third candidate location corresponding to the third candidate entity in the vector space;
- linking both the first mention in the document and the second mention in the document to one candidate entity, comprising the first candidate entity but not the third candidate entity, responsive to:
- determining that a first distance between the third candidate location corresponding to the third candidate entity and the second mention location corresponding to the second mention in the document is less than a second distance between the first candidate location corresponding to the first candidate entity and the second mention location corresponding to the second mention in the document;
- determining that the second distance between the first candidate location corresponding to the first candidate entity and the second mention location corresponding to the second mention in the document is less than a threshold distance;
- and
- determining that a third distance between the first candidate location corresponding to the first candidate entity and the first mention location corresponding to the first mention in the document is less than a fourth distance between the second candidate location corresponding to the second candidate entity and the first mention location corresponding to the first mention in the document;
- determining a topic of the document based upon the linking of the first candidate entity to both the first mention and to the second mention;
- selecting one or more candidate entities based upon at least one of:
- local context corresponding to one or more verbs used in a second document being indicative of actions associated with a first profession associated with the one or more candidate entities;
- or
- locational context determined based upon GPS of a device of a user;
- determining a second topic of the second document based upon the selection of the one or more candidate entities;
- and
- organizing and presenting the document and the second document on a document host based upon the topic of the document and the second topic of the second document.
23 more elements
A non-transitory machine readable medium having stored thereon processor-executable instructions that when executed cause performance of operations, the operations comprising:
- retrieving, by a server, a document from a source via a network connection;
- analyzing the document to identify a plurality of mentions, comprising a first mention and a second mention, in the document;
- processing a plurality of documents extracted from a knowledge base to generate processed documents by transforming one or more hyperlinks in each document to a canonical form for an associated entity;
- training an entity embedding model using the processed documents, wherein a first layer of the entity embedding model models context of entities of the knowledge base and a second layer of the entity embedding model models context of text spans of the knowledge base;
- generating, using the entity embedding model, a graph with a plurality of nodes comprising a first node corresponding to the first mention and a second node corresponding to the second mention;
- assigning a weight to an edge connecting the first node to the second node;
- determining that the first node is assigned a first label corresponding to a first candidate entity;
- randomly traversing the graph, using a traversal logic, to determine whether to label a node and, upon determining not to label the node, determine whether to propagate a label of a neighboring node to the node;
- propagating the first label assigned to the first node to the second node based upon the weight of the edge;
- determining a topic of the document based upon (i) the propagating of the first label assigned to the first node to the second node and (ii) local context corresponding to one or more verbs used in the document being indicative of actions associated with a first profession associated with the first candidate entity;
- retrieving, by the server, a second document from the source via a second network connection;
- analyzing the second document to identify a second plurality of mentions, comprising a third mention and a fourth mention, in the second document;
- generating, using the entity embedding model, a second graph with a second plurality of nodes comprising a third node corresponding to the third mention and a fourth node corresponding to the fourth mention;
- assigning a second weight to a second edge connecting the third node to the fourth node;
- determining that the third node is assigned a third label corresponding to a second candidate entity;
- propagating the third label assigned to the third node to the fourth node based upon the second weight of the second edge;
- determining a second topic of the second document based upon (i) the propagating of the third label assigned to the third node to the fourth node and (ii) second local context corresponding to one or more second verbs used in the document being indicative of actions associated with a second profession associated with the second candidate entity;
- and
- organizing and presenting the document and the second document among a plurality of articles based upon the topic and the second topic.
16 more elements
Dependencies are inferred from the claim text.