Systems and methods for image generation with machine learning models

A system comprising:

• at least one memory storing instructions;
• at least one processor configured to execute the instructions to perform operations for regenerating a region of an image with a machine learning model based on a text input, the operations comprising:
• generating, with a machine learning model trained on a set of images and configured to generate an enhanced image, the enhanced image based on at least one of an input image having a plurality of pixels, a masked image comprising the input image having a masked region, or a text input to the machine learning model;
• wherein the generation of the enhanced image comprises:
• replicating pixel values from the input image or the masked image to the enhanced image;
• generating, with the machine learning model, an image segment based on the text input and the pixel values from the masked image;
• and
• inserting the image segment into the enhanced image by replacing the masked region.

A method comprising:

• generating a masked image by removing pixel values corresponding to a masked region of an input image;
• and
• generating, with a machine learning model trained on a set of images, an enhanced image that is based on the input image by:
• generating an image segment based on a text input and the pixel values corresponding to the masked region;
• and
• inserting the image segment into the enhanced image.

A system comprising:

• at least one memory storing instructions;
• at least one processor configured to execute the instructions to perform operations for generating at least one image, the operations comprising:
• generating, with a machine learning model trained on a set of images, a first enhanced image and a second enhanced image each based on at least one of an input image having a plurality of pixels, a masked image comprising the input image having a masked region, or a text input to the machine learning model;
• wherein the generation of the first and second enhanced images comprises:
• replicating pixel values from the input image or the masked image to the first and second enhanced images;
• generating, with the machine learning model, a first image segment based on the text input and the pixel values from the masked image;
• generating, with the machine learning model, a second image segment based on the text input and the pixel values from the masked image;
• inserting the first image segment into the first enhanced image by replacing the masked region;
• and
• inserting the second image segment into the second enhanced image by replacing the masked region.