Generative object compositing by learning identity-preserving representation

A method of training an image generation model, the method comprising:

• obtaining a first training set including a first training image and a second training image, wherein the second training image depicts an object from the first training image in a different view;
• training, using the first training set during a first training stage, an image generation model to generate a synthetic image that preserves an identity and changes an orientation of an object from an input image;
• obtaining a second training set including a training foreground image, a training background image, and a ground-truth composite image that depicts an object from the training foreground image in a scene from the training background image;
• and
• training, using the second training set during a second training stage, the image generation model to generate a composite image based on an input foreground image and an input background image, wherein the composite image depicts an object from the input foreground image in a scene from the input background image.

An apparatus comprising:

• at least one processor;
• at least one memory including instructions executable by the at least one processor;
• and
• an image generation model comprising parameters stored in the at least one memory and trained to encode a foreground image to obtain a foreground embedding that preserves an identity of an object in the foreground image and generate a composite image based on a background image and the foreground embedding, wherein the composite image depicts the object from the foreground image within a scene from the background image.