Segmentation Guided Image Generation With Adversarial Networks

A method for training an image generator, the method comprising:

• training:
• (i) a generator, implemented with a first neural network, to generate a fake image based on a target segmentation, (ii) a discriminator, implemented with a second neural network, to distinguish a real image from a fake image and output a discrimination result as a function thereof, and (iii) a segmentor, implemented with a third neural network, to generate a segmentation from the fake image, the training including:
• by the generator, outputting the fake image to the discriminator and the segmentor;
• and
• iteratively operating the generator, discriminator, and segmentor during a training period, the iterative operating causing the discriminator and generator to train in an adversarial relationship with each other and the generator and segmentor to train in a collaborative relationship with each other, the generator at the end of the training period having its first neural network trained to generate the fake image based on the target segmentation with more accuracy than at the start of the training period.

A computer program product for training an image generator, the computer program product comprising:

• one or more non-transitory computer-readable storage devices and program instructions stored on at least one of the one or more storage devices, the program instructions, when loaded and executed by a processor, cause an apparatus associated with the processor to:
• train:
• (i) a generator, implemented with a first neural network, to generate a fake image based on a target segmentation, (ii) a discriminator, implemented with a second neural network, to distinguish a real image from a fake image and output a discrimination result as a function thereof, and (iii) a segmentor, implemented with a third neural network, to generate a segmentation from the fake image, the training including:
• by the generator, outputting the fake image to the discriminator and the segmentor;
• and
• iteratively operating the generator, discriminator, and segmentor during a training period, the iterative operating causing the discriminator and generator to train in an adversarial relationship with each other and the generator and segmentor to train in a collaborative relationship with each other, the generator at the end of the training period having its first neural network trained to generate the fake image based on the target segmentation with more accuracy than at the start of the training period.