Term extraction in highly technical domains

A terminology extraction system for fine-tuning of a transformer-based language model by extracting terminology terms from a text document, the system comprising a processor and a memory, communicatively coupled to the processor, wherein the memory stores program code portions that when executed, enable the processor, to

• identify a text snippet in the text document;
• identify candidate multi-word expressions in the identified text snippet using part of speech tags;
• determine a specificity score value for each of the candidate multi-word expressions, wherein the specificity score value is indicative of how unique the candidate multi-word expression is within the text snippet;
• determine a topic similarity score value for each of the candidate multi-word expressions as a distance between an embedding vector corresponding to the identified text snippet and an embedding vector corresponding to the candidate multi-word expression;
• select remaining expressions from the candidate multi-word expressions using a function of a specificity value and a topic similarity value of each of the candidate multi-word expressions;
• add a noun comprised in the text snippet to the remaining expressions depending on a function of a correlation of the noun with a same lemma as one of the remaining expressions and a noun morphology score value of the noun to be added;
• label the remaining multi-word expressions;
• and
• fine-tune an existing pre-trained transformer-based language model using as training data the identified text snippet marked with the labeled remaining expressions.

A computer program product for fine-tuning of a transformer-based language model by extracting terminology terms from a text document, the computer program product comprising a computer readable storage medium having program instructions embodied the therewith, the program instructions being executable by one or more computing systems or controllers to cause the one or more computing systems to:

• identify a text snippet in the text document;
• identify candidate multi-word expressions in the identified text snippet using part of speech tags;
• determine a specificity score value for each of the candidate multi-word expressions, wherein the specificity score value is indicative of how unique the candidate multi-word expression is within the text snippet;
• determine a topic similarity score value for each of the candidate multi-word expressions as a distance between an embedding vector corresponding to the identified text snippet and an embedding vector corresponding to the candidate multi-word expression;
• select remaining expressions from the candidate multi-word expressions using a function of a specificity value and a topic similarity value of each of the candidate multi-word expressions;
• add a noun comprised in the text snippet to the remaining expressions depending on a function of a correlation of the noun with a same lemma as one of the remaining expressions and a noun morphology score value of the noun to be added;
• label the remaining multi-word expressions;
• and
• fine-tune an existing pre-trained transformer-based language model using as training data the identified text snippet marked with the labeled remaining expressions.