Generating images using neural networks
AssigneeDeepMind Technologies Limited
InventorsAaron Gerard Antonius van den Oord, Nal Emmerich Kalchbrenner, Karen Simonyan
Life of this patent
Estimating the term…
Abstract
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating images using neural networks. One of the methods includes generating the output image pixel by pixel from a sequence of pixels taken from the output image, comprising, for each pixel in the output image, generating a respective score distribution over a discrete set of possible color values for each of the plurality of color channels.
Claims20
A neural network system implemented by one or more computers, the neural network system being configured to receive a neural network input and to generate an output image from the neural network input, the output image comprising a plurality of pixels arranged in a two-dimensional map, each pixel having a respective color value for each of a plurality of color channels, and the neural network system comprising:
- one or more initial neural network layers configured to receive the neural network input and to process the neural network input to generate an alternative representation of the neural network input;
- and
- one or more output layers, wherein the output layers are configured to receive the alternative representation and to generate the output image pixel by pixel from a sequence of pixels taken from the output image, comprising, for each pixel in the output image, generating a respective score distribution over a discrete set of possible color values for each of the plurality of color channels.
A system for generating a predicted next video frame that is a prediction of a video frame that will follow a last video frame in a temporal sequence of video frames, wherein the predicted next video frame comprises a plurality of pixels each having a respective color value for each of a plurality of color channels, wherein the pixels are ordered according to a pixel order and the color channels are ordered according to a channel order, and wherein the system comprises:
- an encoder neural network implemented by one or more computers and configured to:
- process each video frame in the temporal sequence to generate an encoded representation of the temporal sequence of video frames;
- and
- a decoder neural network implemented by the one or more computers and configured to, for each color channel of each pixel in the predicted next video frame:
- receive an input comprising (i) the encoded representation of the temporal sequence of video frames, (ii) color values for any pixels in the predicted next video frame before the pixel in the pixel order, and (iii) color values for the pixel for any color channels before the color channel in the channel order;
- and
- process the input to generate an output defining a score distribution over a plurality of possible color values for the color channel of the pixel.
Dependencies are inferred from the claim text.