System using multimodal decorrelated embedding model

A computer-implemented method comprising:

• accessing training data comprising:
• a first plurality of first modality images, and
• a second plurality of second modality images;
• determining, using a first portion of a machine learning network to process at least a portion of the training data, first intermediate embedding data;
• determining, using a second portion of the machine learning network to process the at least a portion of the training data, intersection embedding data and a first loss value, wherein the intersection embedding data is representative of features depicted in a first modality image and a second modality image that are labeled as having a same identity;
• determining, using a third portion of the machine learning network to process the at least a portion of the training data:
• first modality XOR embedding data and an associated second loss value, wherein the first modality XOR embedding data is associated with features in the first modality image and not in the second modality image, and
• second modality XOR embedding data and an associated third loss value, wherein the second modality XOR embedding data is associated with features in the second modality image and not in the first modality image;
• determining, based on the intersection embedding data, the first modality XOR embedding data, and the second modality XOR embedding data, a first set of probability distributions;
• determining, based on the first set of probability distributions, a fourth loss value;
• determining a total loss value based on the first loss value, the second loss value, the third loss value, and the fourth loss value;
• and
• determining trained model data based on the total loss value.

A system comprising:

• a memory, storing first computer-executable instructions;
• and
• a hardware processor to execute the first computer-executable instructions to:
• determine training data comprising:
• a first plurality of labeled first modality images, and
• a second plurality of labeled second modality images;
• determine, using a first portion of a machine learning network to process the first plurality of labeled first modality images and the second plurality of labeled second modality images, first intermediate embedding data;
• determine, using a second portion of the machine learning network to process the first plurality of labeled first modality images and the second plurality of labeled second modality images, intersection embedding data and a first loss value, wherein the intersection embedding data is representative of features depicted in a first modality image and a second modality image that are labeled as having a same identity;
• determine, using a third portion of the machine learning network to process the first plurality of labeled first modality images and the second plurality of labeled second modality images:
• first modality XOR embedding data and an associated second loss value, wherein the first modality XOR embedding data is associated with features in the first modality image and not in the second modality image, and
• second modality XOR embedding data and an associated third loss value, wherein the second modality XOR embedding data is associated with features in the second modality image and not in the first modality image;
• determine, based on the intersection embedding data, the first modality XOR embedding data, and the second modality XOR embedding data, a first set of probability distributions;
• determine, based on the first set of probability distributions, a fourth loss value;
• determine a total loss value based on the first loss value, the second loss value, the third loss value, and the fourth loss value;
• and
• determine trained model data based on the total loss value.