Training multimodal representation learning model on unnanotated multimodal data

A computer-implemented method, comprising:

• receiving, via a processor, unannotated multimodal data;
• estimating, via the processor, a probability of an associated pair of different modalities in the unannotated multimodal data to be correctly associated using a multimodal similarity function and a local density estimation;
• and
• training, via the processor, a multimodal representation learning model on the unannotated multimodal data using the estimated probability to be correctly associated as a weight for each of the sample pairs in a loss function.

A computer program product for training multimodal representation learning models, the computer program product comprising a computer-readable storage medium having program code embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, the program code executable by a processor to cause the processor to:

• receive unannotated multimodal data;
• estimate a probability an associated pair of different modalities in the unannotated multimodal data to be correctly associated using a multimodal similarity function and a local density estimation;
• and
• train a multimodal representation learning model on the unannotated multimodal data using the estimated probability to be correctly associated as a weight for the associated pair in a loss function.