Resource-efficient diffusion models
AssigneeGoogle Inc.
InventorsYang Zhao, Yanwu Xu, Zhisheng Xiao, Tingbo Hou
Life of this patent
Estimating the term…
Abstract
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating a data item by performing a single-step denoising process using a diffusion model neural network. For example, the data items can be images, videos, audio waveforms, sensor outputs, and so on.
Claims20
A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to implement a diffusion model neural network that generates a data item by performing a denoising process, wherein the diffusion model neural network comprises one or more initial neural network blocks followed by one or more intermediate neural network blocks followed by one or more final neural network blocks, and wherein:
- each initial neural network block is configured to process an initial block input having a higher dimensionality to generate an initial block output based on applying a convolution operation to the initial block input;
- each intermediate neural network block is configured to process an intermediate block input having a lower dimensionality that is lower than the higher dimensionality to generate an intermediate block output based on applying the convolution operation and an attention operation to the intermediate block input;
- and
- each final neural network block is configured to process a final block input having the higher dimensionality to generate a final block output based on applying the convolution operation to the final block input.
A method performed by one or more computers, the method comprising:
- generating a data item by performing a denoising process using a diffusion model neural network, wherein the diffusion model neural network comprises one or more initial neural network blocks followed by one or more intermediate neural network blocks followed by one or more final neural network blocks, and wherein the generating comprises:
- processing, by each initial neural network block, an initial block input having a higher dimensionality to generate an initial block output based on applying a convolution operation to the initial block input;
- processing, by each intermediate neural network block, an intermediate block input having a lower dimensionality that is lower than the higher dimensionality to generate an intermediate block output based on applying the convolution operation and an attention operation to the intermediate block input;
- and
- processing, by each final neural network block, a final block input having the higher dimensionality to generate a final block output based on applying the convolution operation to the final block input.
One or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to implement a diffusion model neural network that generates a data item by performing a denoising process, wherein the diffusion model neural network comprises one or more initial neural network blocks followed by one or more intermediate neural network blocks followed by one or more final neural network blocks, and wherein:
- each initial neural network block is configured to process an initial block input having a higher dimensionality to generate an initial block output based on applying a convolution operation to the initial block input;
- each intermediate neural network block is configured to process an intermediate block input having a lower dimensionality that is lower than the higher dimensionality to generate an intermediate block output based on applying the convolution operation and an attention operation to the intermediate block input;
- and
- each final neural network block is configured to process a final block input having the higher dimensionality to generate a final block output based on applying the convolution operation to the final block input.
Dependencies are inferred from the claim text.