High-resolution video generation using image diffusion models

A processor, comprising:

• one or more circuits to:
• update a neural network model to align a plurality of images into frames of a first video by updating at least one layer of the neural network model, wherein the first video has a first spatial resolution, wherein the neural network model comprises a Latent Diffusion Model (LDM) comprising:
• an encoder to map an input from an image space to a latent space;
• and
• a decoder to map latent encoding from the latent space to the image space,
• wherein to update the neural network model, the one or more processing units are to update the decoder according to one or more temporal incoherencies in mapping the latent encoding from the latent space to the image space;
• and
• update an up-sampler neural network model to generate a second video having a second spatial resolution via up-sampling the first video by updating at least one layer of the up-sampler neural network model, wherein the second spatial resolution is higher than the first spatial resolution.

A method, comprising:

• updating a neural network model to align a plurality of images into frames of a first video by updating at least one first temporal attention layer of the neural network model, wherein the first video has a first spatial resolution, wherein the neural network model comprises a Latent Diffusion Model (LDM) comprising:
• an encoder to map an input from an image space to a latent space;
• and
• a decoder to map latent encoding from the latent space to the image space,
• wherein updating the neural network model comprises updating the decoder according to one or more temporal incoherencies in mapping the latent encoding from the latent space to the image space;
• and
• updating an up-sampler neural network model to generate a second video having a second spatial resolution via up-sampling the first video, by updating at least one second temporal attention layer of the up-sampler neural network model, wherein the second spatial resolution is higher than the first spatial resolution.