Computer-based systems for data entity matching detection based on latent similarities in large datasets and methods of use thereof
AssigneeAmerican Express Travel Related Services Company, Inc.
InventorsArpan Somani, Salil Rajeev Joshi, Shourya Roy
Life of this patent
Estimating the term…
Abstract
At least some embodiments are directed to an entity matching detection system. The entity matching detection system includes a latent similarity identification machine learning model that receives one or more data records and generates a final similarity score indicative of a latent similarity between the one or more data records and a second data record. The entity matching detection system can identify lexical and semantic similarities between attribute values and can analyze and compute similarity scores for direct-linked attribute values and cross-linked attribute values extracted from different data records.
Claims20
A method, comprising:
- constructing a hash index database by:
- decomposing a first set of attribute values into a first plurality of N-grams data structures;
- and
- storing a mapping from each of the N-grams data structures of the first plurality of N-grams data structures to one or more attribute values from the first set of attribute values;
- receiving a first string of characters and a second string of characters;
- extracting, from the first string of characters, a second plurality of N-grams data structures;
- querying the hash index database using each of the second plurality N-grams data structures to identify a second set of attribute values based on the mapping;
- extracting, from the second string of characters, a third plurality of N-grams data structures;
- querying the hash index database using each of the third plurality of N-grams data structures to identify a third set of attribute values based on the mapping;
- and
- determining a similarity score between the first string of characters and the second string of characters by applying the second set of attribute values and the third set of attribute values to a machine learning algorithm trained to generate the similarity score based on a comparison of matching attribute values.
A system, comprising:
- a memory storing a hash index database;
- and
- at least one processor coupled to the memory and configured to:
- construct the hash index database by:
- decomposing a first set of attribute values into a first plurality of N-grams data structures;
- and
- storing a mapping from each of the N-grams data structures of the first plurality of N-grams data structures to one or more attribute values from the first set of attribute values;
- receive a first string of characters and a second string of characters;
- extract, from the first string of characters, a second plurality of N-grams data structures;
- query the hash index database using each of the second plurality N-grams data structures to identify a second set of attribute values based on the mapping;
- extract, from the second string of characters, a third plurality of N-grams data structures;
- query the hash index database using each of the third plurality of N-grams data structures to identify a third set of attribute values based on the mapping;
- and
- determine a similarity score between the first string of characters and the second string of characters by applying the second set of attribute values and the third set of attribute values to a machine learning algorithm trained to generate the similarity score based on a comparison of matching attribute values.
A non-transitory computer-readable medium having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform operations comprising:
- constructing a hash index database by:
- decomposing a first set of attribute values into a first plurality of N-grams data structures;
- and
- storing a mapping from each of the N-grams data structures of the first plurality of N-grams data structures to one or more attribute values from the first set of attribute values;
- receiving a first string of characters and a second string of characters;
- extracting, from the first string of characters, a second plurality of N-grams data structures;
- querying the hash index database using each of the second plurality N-grams data structures to identify a second set of attribute values based on the mapping;
- extracting, from the second string of characters, a third plurality of N-grams data structures;
- querying the hash index database using each of the third plurality of N-grams data structures to identify a third set of attribute values based on the mapping;
- and
- determining a similarity score between the first string of characters and the second string of characters by applying the second set of attribute values and the third set of attribute values to a machine learning algorithm trained to generate the similarity score based on a comparison of matching attribute values.
Dependencies are inferred from the claim text.