Cross-domain image analysis and cross-domain image synthesis using deep image-to-image networks and adversarial networks

An apparatus for automatically performing cross-domain based medical image analysis on a medical image of a patient, comprising:

• a processor;
• and
• a memory storing computer program instructions, which when executed by the processor cause the processor to perform operations comprising:
• receiving a medical image of a patient from a first domain;
• inputting the medical image of the patient to a first encoder of a cross-domain deep image-to-image network including the first encoder for the first domain, a second encoder for a second domain, and a decoder;
• and
• automatically generating an output image that provides a result of a target medical image analysis task on the input medical image using the cross-domain deep image-to-image network, by the first encoder converting the input medical image from the first domain to a feature map and the decoder generating the output image from the feature map generated by the first encoder, wherein the first encoder for the first domain is trained together with the second encoder for the second domain at least in part based on a similarity of feature maps generated by the first encoder from training input images from the first domain and feature maps generated by the second encoder from training input images from the second domain, and the decoder is trained to generate output images from the feature maps generated by the first encoder and the feature maps generated by the second encoder.