Modification of objects in film

A computer-implemented method comprising:

• identifying a first instance of an object within input video data comprising a plurality of image frames;
• determining portions of image frames containing the first instance of the object;
• determining, for each of the image frames containing the first instance of the object, corresponding parameter values for a synthetic model of the object;
• and
• modifying, for at least some of the image frames containing the first instance of the object, the determined parameter values for the synthetic model of the object;
• for each determined portion of an image frame containing the first instance of the object:
• rendering a synthetic image of the first instance of the object using the synthetic model and the modified parameter values for the synthetic model;
• generating a composite image comprising at least part of the rendered synthetic image and part of the determined portion of the image frame;
• processing the generated composite images using a deep neural network to generate a second instance of the object, wherein a geometry of the second instance of the object differs from a geometry of the first instance of the object;
• and
• generating output video data by replacing at least part of the first instance of the object in the input video data with a corresponding at least part of the second instance of the object.

One or more non-transient storage media comprising instructions which, when executed by one or more processors, cause the processors to carry out operations comprising:

• identifying a first instance of an object within input video data comprising a plurality of image frames;
• determining portions of image frames containing the first instance of the object;
• determining, for each of the image frames containing the first instance of the object, corresponding parameter values for a synthetic model of the object;
• and
• modifying, for at least some of the image frames containing the first instance of the object, the determined parameter values for the synthetic model of the object;
• for each determined portion of an image frame containing the first instance of the object:
• rendering a synthetic image of the first instance of the object using the synthetic model and the modified parameter values for the synthetic model;
• generating a composite image comprising at least part of the rendered synthetic image and part of the determined portion of the image frame;
• processing the generated composite images using a deep neural network to generate a second instance of the object, wherein a geometry of the second instance of the object differs from a geometry of the first instance of the object;
• and
• generating output video data by replacing at least part of the first instance of the object in the input video data with a corresponding at least part of the second instance of the object.