US 11,106,979 B2 · Unsupervised learning of entity representations using graphs

Abstract

Techniques for implementing a learning semantic representations of sparse entities using unsupervised embeddings are disclosed herein. In some embodiments, a computer system accesses corresponding profile data of users indicating at least one entity of a first facet type associated with the user, and generating a graph data structure comprising nodes and edges based on the accessed profile data, with each node corresponding to a different entity indicated by the accessed profile data, and each edge directly connecting a different pair of nodes and indicating a number of users whose profile data indicates both entities of the pair of nodes. The computer system generating a corresponding embedding vector for the entities based on the graph data structure using an unsupervised machine learning algorithm.

Drawings

Drawings are temporarily unavailable. Open the PDF at USPTO

Drawings

Claims16

Dependent claims

A computer-implemented method comprising:

for each one of a plurality of users of an online service, accessing, by a computer system having a memory and at least one hardware processor, corresponding profile data of the user stored on a database of the online service, the accessed profile data of each user indicating at least one entity associated with the user;
generating, by the computer system, a first graph data structure based on the accessed profile data, the generated first graph data structure comprising a first plurality of nodes and a first plurality of edges, each one of the first plurality of nodes corresponding to a different entity indicated by the accessed profile data, and each one of the first plurality of edges directly connecting a different pair of the first plurality of nodes and indicating a number of the plurality of users whose profile data includes both entities of the pair of nodes that are directly connected by the edge, a first entity of the both entities of the pair of nodes that are directly connected by the edge comprising one of a first job title, a first company, a first skill, a first school, a first educational degree, or a first educational major, and a second entity of the both entities of the pair of nodes that are directly connected by the edge comprising one of a second job title, a second company, second skill, a second school, a second educational degree, or a second educational major;
generating, by the computer system, a corresponding embedding vector for each one of the entities indicated by the accessed profile data based on the first graph data structure using an unsupervised machine learning algorithm, the unsupervised machine learning algorithm is configured to optimize the corresponding embedding vector of each one of the entities to result in a level of similarity between the corresponding embedding vectors of two entities increasing as the number of the plurality of users whose profile data indicates the two entities increases, and the unsupervised machine learning algorithm is further configured to optimize the corresponding embedding vector of each one of the entities to result in the level of similarity between the corresponding embedding vectors of two entities increasing as the number of neighbor nodes shared by the two entities increases;
and
performing, by the computer system, a function of the online service using the generated embedding vectors of the entities.

2 more elements

A system comprising:

at least one hardware processor;
and
a non-transitory machine-readable medium embodying a set of instructions that, when executed by at least one hardware processor, cause the processor to perform operations comprising:
for each one of a plurality of users of an online service, accessing corresponding profile data of the user stored on a database of the online service, the accessed profile data of each user indicating at least one entity associated with the user;
generating a first graph data structure based on the accessed profile data, the generated first graph data structure comprising a first plurality of nodes and a first plurality of edges, each one of the first plurality of nodes corresponding to a different entity indicated by the accessed profile data, and each one of the first plurality of edges directly connecting a different pair of the first plurality of nodes and indicating a number of the plurality of users whose profile data includes both entities of the pair of nodes that are directly connected by the edge, a first entity of the both entities of the pair of nodes that are directly connected by the edge comprising one of a first job title, a first company, a first skill, a first school, a first educational degree, or a first educational major, and a second entity of the both entities of the pair of nodes that are directly connected by the edge comprising one of a second job title, a second company, a second skill, a second school, a second educational degree, or a second educational major, the corresponding entity for each node in the first plurality of nodes of the first graph data structure is of a first facet type;
generating a corresponding embedding vector for each one of the entities indicated by the accessed profile data based on the first graph data structure using an unsupervised machine learning algorithm;
generating a second graph data structure comprising a second plurality of nodes and a second plurality of edges, each one of the second plurality of nodes corresponding to a different entity of a second facet type indicated by the accessed profile data, and each one of the second plurality of edges directly connecting a different pair of the second plurality of nodes and indicating a number of the plurality of users whose profile data indicates both entities of the pair of nodes that are directly connected by the edge in the second graph data structure;
and
generating a corresponding embedding vector for each one of the entities of the second facet type based on the second graph data structure using the unsupervised machine learning algorithm;
and
performing a function of the online service using the generated embedding vectors of the entities, the function of the online service is performed using the generated embedding vectors of the entities of the second facet type.

8 more elements

A non-transitory machine-readable medium embodying a set of instructions that, when executed by at least one hardware processor, cause the processor to perform operations comprising:

for each one of a plurality of users of an online service, accessing corresponding profile data of the user stored on a database of the online service, the accessed profile data of each user indicating at least one entity associated with the user;
generating a first graph data structure based on the accessed profile data, the generated first graph data structure comprising a first plurality of nodes and a first plurality of edges, each one of the first plurality of nodes corresponding to a different entity indicated by the accessed profile data, and each one of the first plurality of edges directly connecting a different pair of the first plurality of nodes and indicating a number of the plurality of users whose profile data includes both entities of the pair of nodes that are directly connected by the edge, a first entity of the both entities of the pair of nodes that are directly connected by the edge comprising one of a first job title, a first company, a first skill, a first school, a first educational degree, or a first educational major, and a second entity of the both entities of the pair of nodes that are directly connected by the edge comprising one of a second job title, a second company, a second skill, a second school, a second educational degree, or a second educational major;
generating a corresponding embedding vector for each one of the entities indicated by the accessed profile data based on the first graph data structure using an unsupervised machine learning algorithm, the unsupervised machine learning algorithm is configured to optimize the corresponding embedding vector of each one of the entities to result in a level of similarity between the corresponding embedding vectors of two entities increasing as the number of the plurality of users whose profile data indicates the two entities increases, and the unsupervised machine learning algorithm is further configured to optimize the corresponding embedding vector of each one of the entities to result in the level of similarity between the corresponding embedding vectors of two entities increasing as the number of neighbor nodes shared by the two entities increases;
and
performing a function of the online service using the generated embedding vectors of the entities.

2 more elements

Dependencies are inferred from the claim text.

Description19,407 words

Summary124 words

#The present application relates generally to architectures for neural networks and, in one specific example, to methods and systems of implementing an architecture for neural networks used for search.

#Current architectures used for processing search queries suffer increased latency in processing search queries that involve complex considerations in generating search results for the search query. In these architectures, the more complex the data of the search query and the data of the items being evaluated for inclusion as search results, the more it is computationally expensive to process the search query while still providing relevant search results. As a result, current search architectures suffer from a technical problem of sacrificing processing speed for search result relevance or vice-versa. Other technical problems arise as well.

Brief description of the drawings14 figures

#Some embodiments of the present disclosure are illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like reference numbers indicate similar elements.

#FIG. 1 is a block diagram illustrating a client-server system, in accordance with an example embodiment.

#FIG. 2 is a block diagram showing the functional components of a social networking service within a networked system, in accordance with an example embodiment.

Detailed description18,969 words

#Example methods and systems of learning semantic representations of sparse entities using unsupervised embeddings via utilization of a graph are disclosed. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of example embodiments. It will be evident, however, to one skilled in the art that the present embodiments may be practiced without these specific details.

#Some or all of the above problems may be addressed by one or more example embodiments disclosed herein. Some technical effects of the system and method of the present disclosure are to improve the ability of a search system to process search queries involving complex data, maximizing the relevance of the search results, while avoiding latency issues that hinder other search systems. Additionally, other technical effects will be apparent from this disclosure as well.

#In some example embodiments, operations are performed by one or more computer systems (or other machines) having a memory and at least one hardware processor, with the operations comprising: for each one of a plurality of users of an online service, accessing, by a computer system having a memory and at least one hardware processor, corresponding profile data of the user stored on a database of the online service, the accessed profile data of each user indicating at least one entity of a first facet type associated with the user; generating, by the computer system, a graph data structure based on the accessed profile data, the generated graph data structure comprising a plurality of nodes and a plurality of edges, each one of the plurality of nodes corresponding to a different entity indicated by the accessed profile data, and each one of the plurality of edges directly, connecting a different pair of the plurality of nodes and indicating a number of the plurality of users whose profile data indicates both entities of the pair of nodes that are directly connected by the edge; generating, by the computer system, a corresponding embedding vector for each one of the entities indicated by the accessed profile data using an unsupervised machine learning algorithm; and performing, by the computer system, a function of the online service using the generated embedding vectors of the entities. In some example embodiments, the online service comprises a social networking service.

Facts

Application

16/021,617Filed 28 Jun 2018

Published

US 2020/0005153 A12 Jan 2020

Granted

31 Aug 2021

Term ends

6 Feb 2039

Estimated · how

20 years from earliest known nonprovisional domestic filing plus recorded patent term adjustment, including 223 days of adjustment. Maintenance-fee lapses, terminal disclaimers and statutory extensions have not been checked. The continuity chain may be incomplete; this is not an enforceability determination.

Classification14 codes

G06F 16/2465
Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries: query processing support for facilitating data mining…
G06N 3/088
Learning methods: non-supervised learning, e.g. competitive learning
G06F 16/248
Querying: presentation of query results
G06F 16/285
Relational databases: clustering or classification

Timeline

28 Jun 2018Application filedUS 16/021,617
26 Sep 2018Assigned to Microsoft Technology Licensing, LLCassignment of assignor's interest
2 Jan 2020PublishedUS 2020/0005153 A1
31 Aug 2021GrantedUS 11,106,979 B2
12 Oct 2026TodayWithin its term
6 Feb 2039Term endsEstimated, includes 223 days of adjustment

Unsupervised learning of entity representations using graphs

Life of this patent

Unsupervised learning of entity representations using graphs

Life of this patent

Life of this patent

Life of this patent

Abstract

Drawings

Drawings

Claims16

Description19,407 words

Drawings

Classification14 codes

Description19,407 words