Automated entity name canonicalization via vector transformation using a database system
AssigneeSalesforce, Inc.
InventorDevrat Agarwal
Life of this patent
Estimating the term…
Abstract
A computing system may process entity name variations. The entity name variations may be transformed into vector representations. The vector representations may be stored in a database. The entity name variations may be clustered into groups of similar entity name variations based on a similarity metric between the vector representations. A canonical identifier for each group of similar entity name variations may be stored in the database. Each canonical identifier identifying a distinct entity.
Claims20
A method comprising:
- preprocessing, using a computing system implemented via a database system, a plurality of entity name variations gathered from a plurality of sources, the preprocessing comprising at least one of cleaning, normalizing, tokenizing, removing stop words, expanding abbreviations, or correcting misspellings;
- transforming the preprocessed entity name variations into vector representations using a word embedding model;
- storing the vector representations in a vector database of the database system, wherein the vector database is optimized to store and index vector data for similarity-based retrieval;
- clustering, based on a similarity metric between the vector representations, the entity name variations into groups of similar entity name variations using an unsupervised clustering algorithm, wherein the clustering discovers the groups from the vector representations without reference to a pre-existing entity name list or pre-assigned canonical identifiers, and wherein the canonical identifier for each group is generated based on the entity name variations within the group;
- and
- generating and storing in the database system a canonical identifier for each group of similar entity name variations, each canonical identifier identifying a distinct entity.
A computing platform implemented using a server system, the computing platform being configurable to cause:
- preprocessing, using a computing system implemented via a database system, a plurality of entity name variations gathered from a plurality of sources, the preprocessing comprising at least one of cleaning, normalizing, tokenizing, removing stop words, expanding abbreviations, or correcting misspellings;
- transforming the preprocessed entity name variations into vector representations using a word embedding model;
- storing the vector representations in a vector database of the database system, wherein the vector database is optimized to store and index vector data for similarity-based retrieval;
- clustering, based on a similarity metric between the vector representations, the entity name variations into groups of similar entity name variations using an unsupervised clustering algorithm, wherein the clustering discovers the groups from the vector representations without reference to a pre-existing entity name list or pre-assigned canonical identifiers, and wherein the canonical identifier for each group is generated based on the entity name variations within the group;
- and
- generating and storing in the database system a canonical identifier for each group of similar entity name variations, each canonical identifier identifying a distinct entity.
A computer program product comprising non-transitory computer-readable program code capable of being executed by one or more processors when retrieved from a non-transitory computer-readable medium, the program code comprising instructions configurable to cause the one or more processors to perform a method comprising:
- preprocessing, using a computing system implemented via a database system, a plurality of entity name variations gathered from a plurality of sources, the preprocessing comprising at least one of cleaning, normalizing, tokenizing, removing stop words, expanding abbreviations, or correcting misspellings;
- transforming the preprocessed entity name variations into vector representations using a word embedding model;
- storing the vector representations in a vector database of the database system, wherein the vector database is optimized to store and index vector data for similarity-based retrieval;
- clustering, based on a similarity metric between the vector representations, the entity name variations into groups of similar entity name variations using an unsupervised clustering algorithm, wherein the clustering discovers the groups from the vector representations without reference to a pre-existing entity name list or pre-assigned canonical identifiers, and wherein the canonical identifier for each group is generated based on the entity name variations within the group;
- and
- generating and storing in the database system a canonical identifier for each group of similar entity name variations, each canonical identifier identifying a distinct entity.
Dependencies are inferred from the claim text.