Contrastive multimodality image registration

A device to perform image registration, the device comprising:

• one or more processors;
• and
• a memory coupled to the one or more processors, with instructions stored thereon that, when executed by the processor, cause the one or more processors to perform operations comprising:
• providing a first image of a first type and a second image of a second type, different from the first type, as input to a trained neural network;
• obtaining, as output of the trained neural network, a displacement field for the first image;
• obtaining a first transformed image by applying the displacement field to the first image via a spatial transform network, wherein corresponding features of the first transformed image and the second image are aligned;
• obtaining, as output of the trained neural network, an inverse displacement field for the second image;
• and
• obtaining a second transformed image by applying the inverse displacement field to the second image via the spatial transform network, wherein corresponding features of the second transformed image and the first image are aligned.

A non-transitory computer-readable medium to train a neural network to perform image registration with instructions stored thereon that, when executed by a processor of a server, cause the processor to perform operations, the operations comprising:

• providing as input to the neural network, a first image and a second image;
• obtaining, using the neural network, a first transformed image based on the first image that is aligned with the second image;
• obtaining, using the neural network, a second transformed image based on the second image that is aligned with the first image;
• computing a loss value based on a comparison of the first transformed image and the second image and a comparison of the second transformed image and the first image;
• and
• adjusting one or more parameters of the neural network based on the loss value.