Controlling hallucinations in generated images

A computing device, comprising:

• a processing device;
• and
• a computer-readable storage medium storing instructions that, responsive to execution by the processing device, causes the processing device to perform operations including:
• receiving, by multiple generative image models, an image set including multiple input images depicting objects and having first backgrounds;
• generating, by each of the multiple generative image models, a generated image set including multiple generated images depicting the objects by replacing the first backgrounds with second backgrounds;
• generating pairs of object masks using a salient object detection model, each pair of object masks including a first object mask of an object within an input image and a second object mask of the object within a corresponding generated image;
• determining, for each pair of object masks associated with each of the multiple generative image models, a hallucination metric based on a comparison of the first object mask and the second object mask;
• and
• selecting a particular generative image model for replacing backgrounds of subsequently received images based on hallucination metrics determined for the pairs of object masks associated with the multiple generative image models.

One or more non-transitory computer-readable storage media storing instructions that, responsive to execution by a processing device, cause the processing device to perform operations including:

• training a generative image model to reduce amounts of deformation introduced into objects depicted in images generated during background replacement;
• and
• during the training, testing improvement of the generative image model at reducing the amounts of deformation, including performing at predefined intervals:
• receiving, by the generative image model, multiple input images depicting objects and having first backgrounds;
• generating, by the generative image model, multiple generated images depicting the objects by replacing the first backgrounds with second backgrounds;
• generating pairs of object masks using a salient object detection model, each pair of object masks including a first object mask of an object within an input image and a second object mask of the object within a corresponding generated image;
• determining, for each pair of object masks, a hallucination metric based on a comparison of the first object mask to the second object mask;
• and
• outputting a degree of improvement of the generative image model based on hallucination metrics determined for the pairs of object masks.