# US 2025/0157008 A1 · Image Generation with Minimal Denoising Diffusion Steps

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0157008 A1
- Document ID: US20250157008A1
- Status: Published application
- Kind: A1
- Application number: 18/949,522
- Filed: 2024-11-15
- Published: 2025-05-15
- Earliest claimed: 2023-11-15
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Tingbo Hou; Yanwu Xu; Yang Zhao; Zhisheng Xiao
- Web page: https://patentgrep.com/patent/US20250157008A1

## CPC

- G06T 5/60: using machine learning, e.g. neural networks
- G06T 5/70: Denoising; Smoothing
- G06T 11/00: Two-dimensional [2D] image generation
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Provided is a one-step text-to-image generative model, which represents a fusion of GAN and diffusion model elements. In particular, despite the promising outcomes of prior diffusion GAN hybrid models, achieving one-step sampling and extending their utility to text-to-image generation remains a complex challenge. The present disclosure provides a number of innovative techniques to enhance diffusion GAN models, resulting in an ultra-fast text-to-image model capable of producing high-quality images in a single sampling step.

## Claims

### Claim 1

A computer-implemented method to train machine learning models, the method comprising:
obtaining, by a computing system comprising one more computing devices, a pre-trained denoising diffusion model comprising a set of pre-trained model parameters;
instantiating, by the computing system, a first instance of the pre-trained denoising diffusion model as a generator model having the set of pre-trained model parameters;
instantiating, by the computing system, a second instance of the pre-trained denoising diffusion model as a discriminator model having the set of pre-trained model parameters; and
finetuning, by the computing system, at least the generator model on a finetuning dataset, wherein finetuning, by the computing system, the generator model comprises modifying, by the computing system, the set of pre-trained model parameters of the generator model based on a generative adversarial network loss term that provides a loss value based on an output of the discriminator model.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein finetuning, by the computing system, the generator model further comprises modifying, by the computing system, the set of pre-trained model parameters of the generator model based on a reconstruction loss term that provides a loss value based on an output of the generator model.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein finetuning, by the computing system, the generator model further comprises finetuning, by the computing system, the generator model on a text-to-image generation task.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, wherein finetuning, by the computing system, the generator model further comprises finetuning, by the computing system, the generator model on a domain-specific downstream task.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein the generator model is configured to receive and process a noise sample to generate a denoised synthetic image in a single denoising step.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein the pre-trained denoising diffusion model comprises a pre-trained latent diffusion model.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein the pre-trained denoising diffusion model comprises a U-Net.

### Claim 8

A computer system comprising one or more computing devices, the computing system configured to perform operations to train image generation models, the operations comprising:
obtaining a training example comprising a training image;
performing one or more forward diffusion steps on the training example to generate a partially noised training example;
performing an additional forward diffusion step on the partially noised training example to generate an additionally noised training example;
processing the additionally noised training example with a generator model to generate fully de-noised prediction, wherein the generator model comprises a denoising diffusion model;
performing one or more forward diffusion steps on the fully de-noised prediction to generate a partially re-noised prediction;
processing the partially re-noised prediction with a discriminator model to generate a discriminator prediction; and
updating one or more parameter values of at least the generator model based on a loss function, wherein the loss function comprises: a reconstruction loss term that generates a reconstruction loss value based on the training example and the fully de-noised prediction, and a GAN loss term that generates a GAN loss value based on the discriminator prediction.

### Claim 9 (depends on 8)

The computer system of claim 8, wherein processing the additionally noised training example with the generator model to generate fully de-noised prediction comprises performing only a single denoising step with the generator model to generate fully de-noised prediction from the additionally noised training example.

### Claim 10 (depends on 8)

The computer system of claim 8, wherein processing the additionally noised training example with the generator model to generate fully de-noised prediction comprises performing a text-to-image generation task on the additionally noised training example and a text prompt.

### Claim 11 (depends on 8)

The computer system of claim 8, wherein the generator model and the discriminator model have both been initialized from a pre-trained diffusion model.

### Claim 12 (depends on 8)

The computer system of claim 8, wherein the generator model and the discriminator model have both been initialized from a pre-trained diffusion model.

### Claim 13 (depends on 8)

The computer system of claim 8, wherein the generator model and the discriminator model have both been initialized from a pre-trained latent diffusion model.

### Claim 14 (depends on 8)

The computer system of claim 8, wherein the generator model and the discriminator model both comprise a U-Net architecture.

### Claim 15 (depends on 8)

The computer system of claim 8, wherein the reconstruction loss term evaluates a KL divergence between the training example and the fully de-noised prediction.

### Claim 16 (depends on 8)

The computer system of claim 8, wherein the operations further comprise updating one or more parameter values of the discriminator model based on a second GAN loss term that generates a second GAN loss value based on the discriminator prediction.

### Claim 17

One or more non-transitory computer-readable media that store a generator model that has been trained by performance of training operations, the training operations comprising:
obtaining, by a computing system comprising one more computing devices, a pre-trained denoising diffusion model comprising a set of pre-trained model parameters;
instantiating, by the computing system, a first instance of the pre-trained denoising diffusion model as the generator model having the set of pre-trained model parameters;
instantiating, by the computing system, a second instance of the pre-trained denoising diffusion model as a discriminator model having the set of pre-trained model parameters; and
finetuning, by the computing system, at least the generator model on a finetuning dataset, wherein finetuning, by the computing system, the generator model comprises modifying, by the computing system, the set of pre-trained model parameters of the generator model based on a generative adversarial network loss term that provides a loss value based on an output of the discriminator model.

### Claim 18 (depends on 17)

The one or more non-transitory computer-readable media of claim 17, wherein finetuning, by the computing system, the generator model further comprises modifying, by the computing system, the set of pre-trained model parameters of the generator model based on a reconstruction loss term that provides a loss value based on an output of the generator model.

### Claim 19 (depends on 17)

The one or more non-transitory computer-readable media of claim 17, wherein finetuning, by the computing system, the generator model further comprises finetuning, by the computing system, the generator model on a text-to-image generation task.

### Claim 20 (depends on 17)

The one or more non-transitory computer-readable media of claim 17, wherein finetuning, by the computing system, the generator model further comprises finetuning, by the computing system, the generator model on a domain-specific downstream task.
