System and method for automated learning from sensors

A non-transitory computer readable medium having a storage that stores instructions, the instructions executed by a processor, the instructions comprising:

• receiving first inputs associated with a first modality, and second inputs associated with a second modality;
• processing the received first inputs and second inputs in a convolutional neural network (CNN), wherein a first set of weights are assigned to the first inputs and a second set of weights are assigned to the second inputs;
• determining a loss for each of the first inputs and the second inputs based on a loss function that applies the first set of weights, the second set of weights, and a presence of a co-occurrence;
• generating a shared feature space as an output of the CNN, wherein a distance between cells associated with the first inputs and the second inputs in the shared feature space is determined based on the loss associated with each of the first inputs and the second inputs;
• and
• based on the shared feature space, providing an output indicative of a co-occurrence or not.

A computer-implemented method, comprising:

• receiving historical pairing information associated with a pairing of a positron emission tomography (PET) image and a first magnetic resonance imaging (MRI) image;
• providing the historical pairing information of the PET image and the first MRI image to a first mirror network and a second mirror network of a neural network, to generate a first output feature and a second output feature for a shared feature space and learning a first set of mapping weights of the first mirror network, to learn a second set of mapping weights of the second mirror network;
• providing a second MM image;
• and
• generating an output of the shared feature space that provides the PET image associated with the second MM image based on the historical pairing information and a loss function.