System and method for recognizing domain specific named entities using domain specific word embeddings

A system comprising:

• one or more processors;
• and
• at least one memory coupled to the one or more processors, and storing instructions that, when executed by the one or more processors, cause the system to perform operations including:
• extracting a number of terms from a text under consideration;
• identifying, among the number of terms, a set of unmatched terms that do not match any of a plurality of known terms previously incorporated into the text incorporation system;
• passing each respective unmatched term of the set of unmatched terms to a vectorization module;
• embedding, using the vectorization module, a vectorized version of each respective unmatched term in a vector space;
• comparing the vectorized version of each respective unmatched term to known term vectors previously embedded in the vector space, each of the known term vectors associated with one of the plurality of known terms;
• passing, to a machine learning model, candidate terms corresponding to known term vectors closest to the vectorized versions of the set of unmatched terms;
• identifying, using the machine learning model, a best candidate term of the candidate terms for each respective unmatched term of the set of unmatched terms;
• mapping the best candidate terms to corresponding unmatched terms in the text under consideration;
• and
• incorporating the text under consideration into the system based on the mappings.

A non-transitory computer-readable medium storing instructions that, when executed by one or more processors of a system, cause the system to perform operations including:

• extracting a number of terms from a text under consideration;
• identifying, among the number of terms, a set of unmatched terms that do not match any of a plurality of known terms previously incorporated into the text incorporation system;
• passing each respective unmatched term of the set of unmatched terms to a vectorization module;
• embedding, using the vectorization module, a vectorized version of each respective unmatched term in a vector space;
• comparing the vectorized version of each respective unmatched term to known term vectors previously embedded in the vector space, each of the known term vectors associated with one of the plurality of known terms;
• passing, to a machine learning model, candidate terms corresponding to known term vectors closest to the vectorized versions of the set of unmatched terms;
• identifying, using the machine learning model, a best candidate term of the candidate terms for each respective unmatched term of the set of unmatched terms;
• mapping the best candidate terms to corresponding unmatched terms in the text under consideration;
• and
• incorporating the text under consideration into the system based on the mappings.