Generating videos using diffusion models

A method of generating an output video, the method comprising:

• initializing a current intermediate representation of the output video, wherein the current intermediate representation comprises a set of latents, wherein the set of latents is sampled from a noise distribution;
• updating the current intermediate representation at each of a plurality of iterations, wherein the updating comprises, at each iteration:
• processing an intermediate input for the iteration comprising the current intermediate representation using a diffusion model that is configured to process the intermediate input to generate a noise output;
• and
• updating the current intermediate representation using the noise output for the iteration;
• and
• generating the output video based on the updated current intermediate representation at a final iteration.

A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations for generating an output video, wherein the operations comprise:

• initializing a current intermediate representation of the output video, wherein the current intermediate representation comprises a set of latents, wherein the set of latents is sampled from a noise distribution;
• updating the current intermediate representation at each of a plurality of iterations, wherein the updating comprises, at each iteration:
• processing an intermediate input for the iteration comprising the current intermediate representation using a diffusion model that is configured to process the intermediate input to generate a noise output;
• and
• updating the current intermediate representation using the noise output for the iteration;
• and
• generating the output video based on the updated current intermediate representation at a final iteration.

One or more non-transitory computer-readable storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations for generating an output video, wherein the operations comprise:

• initializing a current intermediate representation of the output video, wherein the current intermediate representation comprises a set of latents, wherein the set of latents is sampled from a noise distribution;
• updating the current intermediate representation at each of a plurality of iterations, wherein the updating comprises, at each iteration:
• processing an intermediate input for the iteration comprising the current intermediate representation using a diffusion model that is configured to process the intermediate input to generate a noise output;
• and
• updating the current intermediate representation using the noise output for the iteration;
• and
• generating the output video based on the updated current intermediate representation at a final iteration.