Systems and methods for image generation via diffusion

A system for generating an image, the system comprising:

• a memory that stores a diffusion model and a plurality of processor executable instructions;
• a communication interface that receives a natural language prompt;
• and
• one or more hardware processors that read and execute the plurality of processor-executable instructions from the memory to perform operations comprising:
• obtaining a noised vector for generating a visual output corresponding to the natural language prompt;
• generating, by the diffusion model, a denoised vector by a sequence of repeated forward iterations of the diffusion model from the noised vector;
• calculating a gradient of a loss computed from the denoised vector, with respect to the noised vector by backpropagation of the diffusion model;
• updating the noised vector based on the gradient;
• and
• generating a final denoised visual output based on the sequence of repeated forward passes of the diffusion model from the updated noised vector conditioned on the natural language prompt.

A non-transitory machine-readable medium comprising a plurality of machine-executable instructions which, when executed by one or more processors, are adapted to cause the one or more processors to perform operations comprising:

• receiving, via a data interface, a natural language prompt;
• obtaining a noised vector for generating a visual output corresponding to the natural language prompt;
• generating, by a diffusion model, a denoised vector by a sequence of repeated forward iterations of the diffusion model from the noised vector;
• calculating a gradient of a loss computed from the denoised vector, with respect to the noised vector by backpropagation of the diffusion model;
• updating the noised vector based on the gradient;
• and
• generating a final denoised visual output based on the sequence of repeated forward passes of the diffusion model from the updated noised vector conditioned on the natural language prompt.