Iterative multiscale image generation using neural networks

A method of generating an output image having an output resolution, each pixel in the output image having a respective value for each of one or more channels, the method comprising:

• obtaining a low-resolution version of the output image;
• and
• upscaling the low-resolution version of the output image to generate the output image having the output resolution by repeatedly performing the following operations until an image with an output resolution is obtained:
• obtaining a current version of the output image having a current resolution;
• and
• processing the current version of the output image using a set of convolutional neural networks that are specific to the current resolution to generate an updated version of the output image having an updated resolution that is higher than the current resolution,
• wherein the set of convolutional neural networks that are specific to the current resolution comprises:
• a first convolutional neural network that is configured to receive a first input comprising the current version of the image and to generate a first output image that includes columns of pixels from an intermediate version of the output image having an intermediate resolution that is higher than the current resolution but lower than the updated resolution, and
• a second convolutional neural network that is configured to receive a second input comprising the intermediate version of the output image and to generate a second output image that includes rows of pixels from the updated version of the output image.

A method of generating an output image having an output resolution, each pixel in the output image having a respective value for each of one or more channels, the method comprising:

• obtaining a low-resolution version of the output image;
• and
• upscaling the low-resolution version of the output image to generate the output image having the output resolution by repeatedly performing the following operations until an image with an output resolution is obtained:
• obtaining a current version of the output image having a current resolution;
• and
• processing the current version of the output image using a set of convolutional neural networks that are specific to the current resolution to generate an updated version of the output image having an updated resolution that is higher than the current resolution, wherein the set of convolutional neural networks that are specific to the current resolution comprises:
• a first convolutional neural network that is configured to receive a first input comprising the current version of the image and to generate a first output image that includes rows of pixels from an intermediate version of the output image having an intermediate resolution that is higher than the current resolution but lower than the updated resolution, and
• a second convolutional neural network that is configured to receive a second input comprising the intermediate version of the output image and to generate a second output image that includes columns of pixels from the updated version of the output image.

A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations for generating an output image having an output resolution, each pixel in the output image having a respective value for each of one or more channels, the operations comprising:

• obtaining a low-resolution version of the output image;
• and
• upscaling the low-resolution version of the output image to generate the output image having the output resolution by repeatedly performing the following operations until an image with an output resolution is obtained:
• obtaining a current version of the output image having a current resolution;
• and
• processing the current version of the output image using a set of convolutional neural networks that are specific to the current resolution to generate an updated version of the output image having an updated resolution that is higher than the current resolution,
• wherein the set of convolutional neural networks that are specific to the current resolution comprises:
• a first convolutional neural network that is configured to receive a first input comprising the current version of the image and to generate a first output image that includes columns of pixels from an intermediate version of the output image having an intermediate resolution that is higher than the current resolution but lower than the updated resolution, and
• a second convolutional neural network that is configured to receive a second input comprising the intermediate version of the output image and to generate a second output image that includes rows of pixels from the updated version of the output image.

A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations for generating an output image having an output resolution, each pixel in the output image having a respective value for each of one or more channels, the operations comprising:

• obtaining a low-resolution version of the output image;
• and
• upscaling the low-resolution version of the output image to generate the output image having the output resolution by repeatedly performing the following operations until an image with an output resolution is obtained:
• obtaining a current version of the output image having a current resolution;
• and
• processing the current version of the output image using a set of convolutional neural networks that are specific to the current resolution to generate an updated version of the output image having an updated resolution that is higher than the current resolution, wherein the set of convolutional neural networks that are specific to the current resolution comprises:
• a first convolutional neural network that is configured to receive a first input comprising the current version of the image and to generate a first output image that includes rows of pixels from an intermediate version of the output image having an intermediate resolution that is higher than the current resolution but lower than the updated resolution, and
• a second convolutional neural network that is configured to receive a second input comprising the intermediate version of the output image and to generate a second output image that includes columns of pixels from the updated version of the output image.

One or more non-transitory computer storage media storing instructions that, when executed by one or more computers, cause the one or more computers to perform operations for generating an output image having an output resolution, each pixel in the output image having a respective value for each of one or more channels, the operations comprising:

• obtaining a low-resolution version of the output image;
• and
• upscaling the low-resolution version of the output image to generate the output image having the output resolution by repeatedly performing the following operations until an image with an output resolution is obtained:
• obtaining a current version of the output image having a current resolution;
• and
• processing the current version of the output image using a set of convolutional neural networks that are specific to the current resolution to generate an updated version of the output image having an updated resolution that is higher than the current resolution,
• wherein the set of convolutional neural networks that are specific to the current resolution comprises:
• a first convolutional neural network that is configured to receive a first input comprising the current version of the image and to generate a first output image that includes columns of pixels from an intermediate version of the output image having an intermediate resolution that is higher than the current resolution but lower than the updated resolution, and
• a second convolutional neural network that is configured to receive a second input comprising the intermediate version of the output image and to generate a second output image that includes rows of pixels from the updated version of the output image.

One or more non-transitory computer storage media storing instructions that, when executed by one or more computers, cause the one or more computers to perform operations for generating an output image having an output resolution, each pixel in the output image having a respective value for each of one or more channels, the operations comprising:

• obtaining a low-resolution version of the output image;
• and
• upscaling the low-resolution version of the output image to generate the output image having the output resolution by repeatedly performing the following operations until an image with an output resolution is obtained:
• obtaining a current version of the output image having a current resolution;
• and
• processing the current version of the output image using a set of convolutional neural networks that are specific to the current resolution to generate an updated version of the output image having an updated resolution that is higher than the current resolution, wherein the set of convolutional neural networks that are specific to the current resolution comprises:
• a first convolutional neural network that is configured to receive a first input comprising the current version of the image and to generate a first output image that includes rows of pixels from an intermediate version of the output image having an intermediate resolution that is higher than the current resolution but lower than the updated resolution, and
• a second convolutional neural network that is configured to receive a second input comprising the intermediate version of the output image and to generate a second output image that includes columns of pixels from the updated version of the output image.