Systems and methods for motion correction in synthetic images

A system for generating synthetic video comprising:

• one or more memory units storing instructions;
• and
• one or more processors configured to execute the instructions to perform operations comprising:
• generating background difference images from a sequence of images;
• generating synthetic difference images using an image sequence generator model;
• determining, for individual synthetic difference images, a location of a respective synthetic object movement edge;
• generating a sequence of adjusted difference images based on the synthetic difference images and the locations of the respective synthetic object movement edges;
• generating merged difference images based on the adjusted difference images and the background difference images;
• and
• generating texturized images based on the merged difference images.

A method for generating synthetic video comprising:

• generating background difference images based on a sequence of images;
• generating synthetic difference images using an image sequence generator model;
• determining, for individual synthetic difference images, a location of a respective synthetic object movement edge;
• generating a sequence of adjusted difference images based on the synthetic difference images and the locations of the respective synthetic object movement edges;
• generating merged difference images based on the adjusted difference images and the background difference images;
• and
• generating texturized images based on the merged difference images.

A system for generating synthetic video comprising:

• one or more memory units storing instructions;
• and
• one or more processors configured to execute the instructions to perform operations comprising:
• training an image sequence generator model to generate synthetic difference images based on training difference images.
• generating background difference images based on a sequence of images and a method of image subtraction;
• determining, for individual background difference images, a location of a respective background movement object edge;
• generating synthetic difference images using the image sequence generator model by iteratively accepting a previous synthetic difference image as an input and returning a subsequent synthetic difference image as an output, starting from a first difference image;
• determining, for individual synthetic difference images, a location of a respective synthetic object movement edge;
• generating a static background image based on the sequence of images;
• determining the location of a reference edge in the static background image using at least one of an edge detecting method, an object detection method, a homography method, or a pose estimation method;
• generating a sequence of adjusted difference images based on the synthetic difference images, the locations of the respective synthetic object movement edges, and the location of the reference edge;
• generating merged difference images based on the adjusted difference images, the background difference images, and the locations of the background movement edges;
• and
• generating texturized images based on the adjusted difference images, wherein generating at least one of the texturized images comprises implementing a decoder model to perform at least one of a forward-step decoding or backward-step decoding.