Compact entity identifier embeddings
AssigneeMicrosoft Technology Licensing, LLC
InventorsYiming Ma, Xuhong Zhang, Wei Lu, Mingzhou Zhou
Life of this patent
Estimating the term…
Abstract
The disclosed embodiments provide a system for processing data. During operation, the system applies a first set of hash functions to a first entity identifier (ID) for a first entity to generate a first set of hash values. Next, the system produces a first set of intermediate vectors from the first set of hash values and a first set of lookup tables by matching each hash value in the first set of hash values to an entry in a corresponding lookup table in the first set of lookup tables. The system then performs an element-wise aggregation of the first set of intermediate vectors to produce a first embedding. Finally, the system outputs the first embedding for use by a machine learning model.
Claims20
A method, comprising:
- applying, by one or more computer systems, a first set of hash functions to a first entity identifier (ID) for a first entity to generate a first set of hash values;
- using the first set of hash values as indexes, retrieving, from a first set of lookup tables, a first set of intermediate vectors that represent the first entity ID, wherein the first set of intermediate vectors has a dimensionality comprising a common number of dimensions associated with the first entity and a dimension comprises a numeric value;
- aggregating corresponding dimensions of the first set of intermediate vectors using an element-wise aggregation to produce a first embedding that represents the first entity ID and has the same dimensionality as the first set of intermediate vectors;
- applying a machine learning model to an embedding pair comprising the first embedding and a second embedding that represents a second entity ID, wherein the second embedding is produced by element-wise aggregation of a second set of intermediate vectors retrieved from a second set of lookup tables using a second set of hash values as indexes and has the same dimensionality as the first embedding;
- based on the embedding pair, generating, by the machine learning model, output that represents a likelihood value, wherein the likelihood value is associated with a positive label as an indication of a positive interaction between the first entity and a second entity associated with the second entity ID;
- and
- training the machine learning model to predict an outcome associated with the first entity and the second entity based on the first embedding and an additional input.
A system comprising:
- one or more processors;
- and
- memory storing instructions that, when executed by the one or more processors, cause the system to:
- using a first set of hash values as indexes, retrieve, from a first set of lookup tables, a first set of intermediate vectors that represent a first entity identifier (ID), wherein the first set of intermediate vectors has a dimensionality comprising a common number of dimensions associated with the first entity and a dimension comprises a numeric value;
- aggregate corresponding dimensions of the first set of intermediate vectors using an element-wise aggregation to produce a first embedding that represents the first entity ID and has the same dimensionality as the first set of intermediate vectors;
- apply a machine learning model to an embedding pair comprising the first embedding and a second embedding that represents a second entity ID, wherein the second embedding is produced by element-wise aggregation of a second set of intermediate vectors retrieved from a second set of lookup tables using a second set of hash values as indexes and has the same dimensionality as the first embedding;
- based on the embedding pair, generate, by the machine learning model, output that represents a likelihood value, wherein the likelihood value is associated with a positive label as an indication of a positive interaction between the first entity and a second entity associated with the second entity ID;
- and
- train the machine learning model to predict an outcome associated with the first entity and the second entity based on the first embedding and an additional input.
A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform operations comprising:
- applying, by one or more computer systems, a first set of hash functions to a first entity identifier (ID) for a first entity to generate a first set of hash values;
- using the first set of hash values as indexes, retrieving, from a first set of lookup tables, a first set of intermediate vectors that represent the first entity ID, wherein the first set of intermediate vectors has a dimensionality comprising a common number of dimensions associated with the first entity and a dimension comprises a numeric value;
- aggregating corresponding dimensions of the first set of intermediate vectors using an element-wise aggregation to produce a first embedding that represents the first entity ID and has the same dimensionality as the first set of intermediate vectors;
- applying a machine learning model to an embedding pair comprising the first embedding and a second embedding that represents a second entity ID, wherein the second embedding is produced by element-wise aggregation of a second set of intermediate vectors retrieved from a second set of lookup tables using a second set of hash values as indexes and has the same dimensionality as the first embedding;
- based on the embedding pair, generating, by the machine learning model, output that represents a likelihood value, wherein the likelihood value is associated with a positive label as an indication of a positive interaction between the first entity and a second entity associated with the second entity ID;
- and
- training the machine learning model to predict an outcome associated with the first entity and the second entity based on the first embedding and an additional input.
Dependencies are inferred from the claim text.