Back to results
US 11,386,276 B2Granted
Method and system for language and domain acceleration with embedding alignment
AssigneeInternational Business Machines Corporation
InventorsXiaotong Liu, Anbang Xu, Yingbei Tong, Rama Kalyani T. Akkiraju
Life of this patent
Estimating the term…
Abstract
A method, system and a computer program product are provided for aligning embeddings of multiple languages and domains into a shared embedding space by transforming monolingual embeddings into a multilingual embeddings in a first shared embedding space using a cross-lingual learning process, and then transforming the multilingual embeddings into cross-domain, multilingual embeddings in a second shared embedding space using a cross-domain learning process.
Claims15
A method of aligning embeddings of multiple languages and domains into a shared embedding space, comprising:
- obtaining, by an information handling system comprising a processor and a memory, a plurality of monolingual embeddings trained in a plurality of languages and domains using a data pre-processing process;
- constructing, by the information handling system, a first parallel vocabulary from the plurality of monolingual embeddings;
- extracting, by the information handling system, a plurality of unigrams from the plurality of monolingual embeddings;
- sorting, by the information handling system, the plurality of unigrams in descending order of frequency of appearance in the plurality of monolingual embeddings;
- applying, by the information handling system, a machine translation process to translate words in the plurality of unigrams from a source language to a target language and from the target language to the source language;
- storing, by the information handling system, translation pairs that exist in both directions between the source language and the target language as the first parallel vocabulary;
- computing, by the information handling system, a linear transformation using the parallel vocabulary as anchor points to generate a transformation matrix between each of the plurality of monolingual embeddings and the first shared embedding space;
- applying, by the information handling system, the transformation matrix to each of the plurality of monolingual embeddings to generate the plurality of multilingual embeddings in the first shared embedding space;
- and
- transforming, by the information handling system, the plurality of multilingual embeddings into a plurality of cross-domain, multilingual embeddings in a second shared embedding space with a cross-domain learning process which uses a linear transformation based on a second parallel vocabulary constructed between multiple domains, thereby aligning multiple embeddings of different languages and domains using both continuous embedding training and parallel vocabulary-based linear transformation.
7 more elements
An information handling system comprising:
- one or more processors;
- a memory coupled to at least one of the processors;
- a set of instructions stored in the memory and executed by at least one of the processors to align embeddings of multiple languages and domains into a shared embedding space, wherein the set of instructions are executable to perform actions of:
- receiving, by an information handling system comprising a processor and a memory, a plurality of input text data files;
- identifying, by the information handling system, a language for each of the plurality of input text data files;
- identifying, by the information handling system, a domain for each of the plurality of input text data files;
- training, by the information handling system, a plurality of monolingual embeddings for each language and each domain identified from the plurality of input text data files;
- extracting, by the information handling system, a plurality of unigrams from the plurality of monolingual embeddings;
- sorting, by the information handling system, the plurality of unigrams in descending order of frequency of appearance in the plurality of monolingual embeddings;
- applying, by the information handling system, a machine translation process to translate words in the plurality of unigrams from a source language to a target language and from the target language to the source language;
- storing, by the information handling system, translation pairs that exist in both directions between the source language and the target language as a first parallel vocabulary;
- computing, by the information handling system, a linear transformation using the first parallel vocabulary as anchor points to generate a transformation matrix between each of the plurality of monolingual embeddings and the first shared embedding space;
- applying, by the information handling system, the transformation matrix to each of the plurality of monolingual embeddings to generate the plurality of multilingual embeddings in the first shared embedding space;
- and
- transforming, by the information handling system, the plurality of multilingual embeddings into a plurality of cross-domain, multilingual embeddings in a second shared embedding space with a cross-domain learning process which uses a linear transformation based on a second parallel vocabulary constructed between multiple domains, thereby aligning multiple embeddings of different languages and domains using both continuous embedding training and parallel vocabulary-based linear transformation.
12 more elements
A computer program product stored in a computer readable storage medium, comprising computer instructions that, when executed by a processor at an information handling system, causes the system to align embeddings of multiple languages and domains into a shared embedding space by:
- receiving, by the processor, a plurality of monolingual embeddings trained in a plurality of languages and domains using a data pre-processing process;
- extracting, by the processor, a plurality of unigrams from the plurality of monolingual embeddings;
- sorting, by the processor, the plurality of unigrams in descending order of frequency of appearance in the plurality of monolingual embeddings;
- applying, by the processor, a machine translation process to translate words in the plurality of unigrams from a source language to a target language and from the target language to the source language;
- storing, by the processor, translation pairs that exist in both directions between the source language and the target language as a first parallel vocabulary;
- computing, by the processor, a linear transformation using the first parallel vocabulary as anchor points to generate a transformation matrix between each of the plurality of monolingual embeddings and the first shared embedding space;
- applying, by the processor, the transformation matrix to each of the plurality of monolingual embeddings to generate the plurality of multilingual embeddings in the first shared embedding space;
- and
- transforming, by the processor, the plurality of multilingual embeddings into a plurality of cross-domain, multilingual embeddings in a second shared embedding space with a cross-domain learning process which uses a linear transformation based on a second parallel vocabulary constructed between multiple domains, thereby aligning multiple embeddings of different languages and domains using both continuous embedding training and parallel vocabulary-based linear transformation.
6 more elements
Dependencies are inferred from the claim text.