Training image-to-image translation neural networks

A system comprising:

• data processing hardware;
• and
• memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising:
• obtaining a source training image in a source domain;
• selecting, from the source training image, a first source patch and a second source patch each representative of respective portions of the source training image;
• translating, using a forward generator neural network, the source training image to a translated target image in a target domain according to parameters of the forward generator neural network, the translated target image comprising a third source patch and a fourth source patch each representative of respective portions of the translated target image, the third source patch and the fourth source patch of the translated target image corresponding to the first source patch and the second source patch of the source training image;
• determining a first energy distance between the first source patch and the second source patch of the source training image;
• determining a second energy distance between the third source patch and the fourth source patch of the translated target image;
• generating a harmonic loss based on the first energy distance and the second energy distance;
• and
• updating, using the harmonic loss, the parameters of the forward generator neural network.