Generating cross-domain data using variational mapping between embedding spaces

A computer program product for learning mapping information between different modalities of data, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method comprising:

• mapping, by a processor of the computer, high-dimensional modalities of data into a low-dimensional manifold to obtain therefor respective low-dimensional embeddings through at least a part of a first network;
• projecting, by the processor, each of the respective low-dimensional embeddings to a common latent space to obtain therefor a respective one of separate latent space distributions in the common latent space through at least a part of a second network;
• optimizing, by the processor, parameters of each of the networks by minimizing a distance between the separate latent space distributions in the common latent space using a variational lower bound;
• and
• outputting, by the processor, the parameters as the mapping information.

A system for learning mapping information between different modalities of data, comprising:

• a processor, configured to
• map high-dimensional modalities of data into a low-dimensional manifold to obtain therefor respective low-dimensional embeddings through at least a part of a first network;
• project each of the respective low-dimensional embeddings to a common latent space to obtain therefor a respective one of separate latent space distributions in the common latent space through at least a part of a second network;
• optimize parameters of each of the networks by minimizing a distance between the separate latent space distributions in the common latent space using a variational lower bound;
• and
• output the parameters as the mapping information.