Methods and systems for prompt-based image editing

A non-transitory computer-readable storage medium storing instructions which, when executed by a processing device, cause the processing device to perform a method, the method comprising:

• accessing an image;
• receiving a selected area of the image;
• receiving a prompt, wherein the prompt is indicative of an editing instruction;
• generating a latent by transforming the image into visual noise;
• predicting, based on the latent and the prompt, a noise image corresponding to the latent;
• subtracting at least a portion of the noise image from the latent to generate an updated latent;
• generating a noisy representation of the image;
• generating a masked latent based on the noisy representation, the updated latent and selected area, wherein the masked latent comprises the updated latent in the areas corresponding to the selected area, and the noisy representation in the areas that do not correspond to the selected area.