Multi-Resolution Training for Latent Diffusion Models
AssigneeGDM Holding LLC
InventorsSander Etienne Lea Dieleman, Hyunjik Kim
Life of this patent
Estimating the term…
Abstract
Provided are systems and methods for training a latent diffusion model that involves two primary stages: training an autoencoder on lower-resolution images and then training a denoising diffusion model on higher-resolution images. As one example, the autoencoder can be trained on images with a resolution of 256×256 pixels or smaller, and subsequently, the diffusion model can be trained on images with a resolution of 512×512 pixels or larger (e.g., megapixel images such as 1024×1024 or larger).
Claims20
A computer-implemented method to train a latent diffusion model, the method comprising:
- training, by a computing system comprising one or more computing devices, an autoencoder model with a plurality of autoencoder training images, wherein the autoencoder model comprises an encoder model configured to generate a latent representation of an input image within a latent space and a decoder model configured to generate a reconstruction of the input image based on the latent representation of the input image generated by the encoder model, and wherein the plurality of autoencoder training images have a first resolution;
- after training, by the computing system, the autoencoder model based on the plurality of autoencoder training images, training, by the computing system, a denoising diffusion model with a plurality of diffusion model training images, wherein the denoising diffusion model is trained within the latent space of the autoencoder, wherein the plurality of diffusion model training images have a second resolution, and wherein the second resolution is greater than the first resolution;
- and
- after training, by the computing system, the denoising diffusion model, outputting, by the computing system, at least the decoder model and the denoising diffusion model as the latent diffusion model.
A computing system comprising a latent diffusion model that has previously been trained by the performance of training operations, the training operations comprising:
- training, by a computing system comprising one or more computing devices, an autoencoder model with a plurality of autoencoder training images, wherein the autoencoder model comprises an encoder model configured to generate a latent representation of an input image within a latent space and a decoder model configured to generate a reconstruction of the input image based on the latent representation of the input image generated by the encoder model, and wherein the plurality of autoencoder training images have a first resolution;
- after training, by the computing system, the autoencoder model based on the plurality of autoencoder training images, training, by the computing system, a denoising diffusion model with a plurality of diffusion model training images, wherein the denoising diffusion model is trained within the latent space of the autoencoder, wherein the plurality of diffusion model training images have a second resolution, and wherein the second resolution is greater than the first resolution;
- and
- after training, by the computing system, the denoising diffusion model, outputting, by the computing system, at least the decoder model and the denoising diffusion model as the latent diffusion model.
Dependencies are inferred from the claim text.