Automatic NER dictionary generation from structured business data

A computerized method performed by one or more processors, the method comprising:

• identifying a set of structured data for use in generating a dictionary for a named entity extraction process;
• identifying a particular collection within the identified structured data to use in the dictionary generation, wherein the particular identified collection includes a plurality of values;
• for each value in the particular identified collection, identifying at least one variant of the value, wherein each of the at least one variant of the value includes at least a portion of the underlying value, wherein the value includes at least one delimiter, and wherein identifying the at least one variant of the value includes:
• identifying a first variant based on the full value;
• identifying a second variant by removing a first suffix from the full value to the first delimiter;
• and
• identifying additional variants by removing additional suffixes for each additional delimiter;
• determining a set of unique variants from the identified at least one variant across each of the values in the particular identified subset;
• and
• adding the determined set of unique variants to the generated dictionary, wherein each unique variant added to the generated dictionary is associated with the value from which the unique variant was derived.

A system comprising:

• one or more processors;
• and
• a computer-readable medium storing instructions executable by the one or more processors to perform operations comprising:
• identifying a set of structured data for use in generating a dictionary for a named entity extraction process;
• identifying a particular collection within the identified structured data to use in the dictionary generation, wherein the particular identified collection includes a plurality of values;
• for each value in the particular identified collection, identifying at least one variant of the value, wherein each of the at least one variant of the value includes at least a portion of the underlying value, wherein the value includes at least one delimiter, and wherein identifying the at least one variant of the value includes:
• identifying a first variant based on the full value;
• identifying a second variant by removing a first suffix from the full value to the first delimiter;
• and
• identifying additional variants by removing additional suffixes for each additional delimiter;
• determining a set of unique variants from the identified at least one variant across each of the values in the particular identified subset;
• and
• adding the determined set of unique variants to the generated dictionary, wherein each unique variant added to the generated dictionary is associated with the value from which the unique variant was derived.