Embedding transformation method and system

A system for performing embedding transformation between multiple sources and targets in three or more languages using one transformation model, the system comprising:

• one or more processors;
• and
• a memory configured to store one or more instructions,
• wherein the one or more processors are configured to execute the stored one or more instructions to:
• obtain a pair of source-side data and first target-side data, the source-side data being text data in a first language and the first target-side data being text data in a second language,
• transform the source-side data into a first embedding vector located in a source-side embedding space of the first language through a source-side embedding model,
• transform the first embedding vector into a second embedding vector located in a target-side embedding space of the second language through a transformation model,
• decode the second embedding vector through a target-side decoder, wherein the target-side decoder is a neural network configured to decode the second embedding vector into second target-side data in the second language;
• train the target-side decoder using a difference between a result of decoding of the second embedding vector and the first target-side data,;
• and
• perform a target task using a trained transformation model, the target task comprising machine translation task,
• wherein the transformation model comprises an attention layer for the first embedding vector and an implicit layer,
• wherein the transformation model is configured to receive the first embedding vector as input and output the second embedding vector, and the implicit layer is configured to repeatedly perform a layer operation based on a value of a weight parameter of the implicit layer using an output vector as input until a preset condition is satisfied,
• wherein the transformation model is trained by using, for each of a combination of multiple sources and targets in three or more languages, a paired dataset comprising pairs of source-side data and target-side data, to learn at least one of a one-to-many, a many-to-one, or a many-to-many embedding relationship between each combination of a source in one of the three or more languages and a target in another one of the three or more languages, and
• wherein in the transforming the source-side data and the transforming the first embedding vector, source information is provided to the source-side embedding model and/or target information is provided to the transformation model.

A non-transitory computer-readable recording medium storing computer program for performing embedding transformation between multiple sources and targets in three or more languages using one transformation model, the computer program being executable by at least one processor to execute:

• obtaining a pair of source-side data and first target-side data, the source-side data being text data in a first language and the first target-side data being text data in a second language;
• transforming the source-side data into a first embedding vector located in a source-side embedding space of the first language through a source-side embedding model;
• transforming the first embedding vector into a second embedding vector located in a target-side embedding space of the second language through a transformation model;
• decoding the second embedding vector through a target-side decoder, wherein the target-side decoder is a neural network configured to decode the second embedding vector into second target-side data in the second language;
• and
• training the target-side decoder using a difference between a result of decoding of the second embedding vector and the first target-side data;
• and
• performing a target task using a trained transformation model, the target task comprising a machine translation task,
• wherein the transformation model comprises an attention layer for the first embedding vector and an implicit layer,
• wherein the transformation model is configured to receive the first embedding vector as input and output the second embedding vector, and the implicit layer is configured to repeatedly perform a layer operation based on a value of a weight parameter of the implicit layer using an output vector as input until a preset condition is satisfied,
• wherein the transformation model is trained by using, for each of a combination of multiple sources and targets in three or more languages, a paired dataset comprising pairs of source-side data and target-side data, to learn at least one of a one-to-many, a many-to-one, or a many-to-many embedding relationship between each combination of a source in one of the three or more languages and a target in another one of the three or more languages, and
• wherein in the transforming the source-side data and the transforming the first embedding vector, source information is provided to the source-side embedding model and/or target information is provided to the transformation model.