Deep neural network for matching entities in semi-structured data

A method comprising:

• accessing training data, the training data comprising plurality of tables with entities having labels identifying partial or complete entity matches, each entity comprising a plurality of fields, each field comprising a field identification and a value;
• tokenizing values in the plurality of tables into one or more training tokens;
• passing each of the one or more training tokens to a trained embedding machine learned model, producing embeddings comprising coordinates for each of the one or more training tokens, wherein an embedding is a set of coordinates in an n-dimensional space;
• concatenating the embeddings for the one or more training tokens into a first matrix;
• constructing a field-of-origin sequence for a first entity of the entities, the field-of-origin sequence comprising a combination of the field identifications of the one or more fields in which the values that were tokenized are stored;
• passing each field identification in the field-of-origin sequence into the embedding machine learned model to generate an embedding for each field identification in the field-of-origin sequence;
• concatenating the embeddings for each field identification in the field-of-origin sequence into a second matrix;
• stacking the first and second matrix to create a third matrix;
• and
• jointly training an attend portion of a decomposable attention neural network, to align elements of the first and second matrix using neural attention, and a compare portion of the decomposable attention neural network, to compare the first entity to a second entity, of the entities, represented by its own matrix of embeddings and to determine if the third matrix matches the matrix of embeddings representing the second entity based on the comparing.

A non-transitory machine-readable medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform operations comprising:

• accessing training data, the training data comprising plurality of tables with entities having labels identifying partial or complete entity matches, each entity comprising a plurality of fields, each field comprising a field identification and a value;
• tokenizing values in the plurality of tables into one or more training tokens;
• passing each of the one or more training tokens to a trained embedding machine learned model, producing embeddings comprising coordinates for each of the one or more training tokens, wherein an embedding is a set of coordinates in an n-dimensional space;
• concatenating the embeddings for the one or more training tokens into a first matrix;
• constructing a field-of-origin sequence for a first entity of the entities, the field-of-origin sequence comprising a combination of the field identifications of the one or more fields in which the values that were tokenized are stored;
• passing each field identification in the field-of-origin sequence into the embedding machine learned model to generate an embedding for each field identification in the field-of-origin sequence;
• concatenating the embeddings for each field identification in the field-of-origin sequence into a second matrix;
• stacking the first and second matrix to create a third matrix;
• and
• jointly training an attend portion of a decomposable attention neural network, to align elements of the first and second matrix using neural attention, and a compare portion of the decomposable attention neural network, to compare the first entity to a second entity, of the entities, represented by its own matrix of embeddings and to determine if the third matrix matches the matrix of embeddings representing the second entity based on the comparing.