Generating images using sparse representations
AssigneeGDM Holding LLC
InventorsCharlie Thomas Curtis Nash, Peter William Battaglia
Life of this patent
Estimating the term…
Abstract
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating compressed representations of synthetic images. One of the methods is a method of generating a synthetic image using a generative neural network, and includes: generating, using the generative neural network, a plurality of coefficients that represent the synthetic image after the synthetic image has been encoded using a lossy compression algorithm; and decoding the synthetic image by applying the lossy compression algorithm to the plurality of coefficients.
Claims20
A method of processing an input sequence using a generative neural network to generate an output sequence having a plurality of output elements, wherein:
- the generative neural network comprises an encoder subnetwork and a sequence of a plurality of decoder subnetworks,
- the encoder subnetwork comprises one or more self-attention layers,
- each decoder subnetwork in the sequence of decoder subnetworks comprises i) one or more self-attention layers and ii) one or more encoder-decoder self-attention layers,
- the method comprising:
- processing an encoder subnetwork input comprising the input sequence using the encoder subnetwork to generate an embedding of the input sequence;
- and
- at each of a plurality of time steps:
- processing, using the first decoder subnetwork in the sequence of decoder subnetworks, a first decoder subnetwork input generated from i) the embedding of the input sequence and ii) a plurality of output tokens generated at previous time steps to generate a first decoder subnetwork output;
- for each subsequent decoder subnetwork in the sequence of decoder subnetworks:
- processing, using the subsequent decoder subnetwork, a subsequent decoder subnetwork input generated from i) the embedding of the input sequence and ii) the decoder subnetwork output generated by the previous decoder subnetwork in the sequence of decoder subnetworks to generate a subsequent decoder subnetwork output;
- and
- generating a new output element in the output sequence using the decoder subnetwork outputs.
A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations for processing an input sequence using a generative neural network to generate an output sequence having a plurality of output elements, wherein:
- the generative neural network comprises an encoder subnetwork and a sequence of a plurality of decoder subnetworks,
- the encoder subnetwork comprises one or more self-attention layers,
- each decoder subnetwork in the sequence of decoder subnetworks comprises i) one or more self-attention layers and ii) one or more encoder-decoder self-attention layers,
- the operations comprising:
- processing an encoder subnetwork input comprising the input sequence using the encoder subnetwork to generate an embedding of the input sequence;
- and
- at each of a plurality of time steps:
- processing, using the first decoder subnetwork in the sequence of decoder subnetworks, a first decoder subnetwork input generated from i) the embedding of the input sequence and ii) a plurality of output tokens generated at previous time steps to generate a first decoder subnetwork output;
- for each subsequent decoder subnetwork in the sequence of decoder subnetworks:
- processing, using the subsequent decoder subnetwork, a subsequent decoder subnetwork input generated from i) the embedding of the input sequence and ii) the decoder subnetwork output generated by the previous decoder subnetwork in the sequence of decoder subnetworks to generate a subsequent decoder subnetwork output;
- and
- generating a new output element in the output sequence using the decoder subnetwork outputs.
One or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations for processing an input sequence using a generative neural network to generate an output sequence having a plurality of output elements, wherein:
- the generative neural network comprises an encoder subnetwork and a sequence of a plurality of decoder subnetworks,
- the encoder subnetwork comprises one or more self-attention layers,
- each decoder subnetwork in the sequence of decoder subnetworks comprises i) one or more self-attention layers and ii) one or more encoder-decoder self-attention layers,
- the operations comprising:
- processing an encoder subnetwork input comprising the input sequence using the encoder subnetwork to generate an embedding of the input sequence;
- and
- at each of a plurality of time steps:
- processing, using the first decoder subnetwork in the sequence of decoder subnetworks, a first decoder subnetwork input generated from i) the embedding of the input sequence and ii) a plurality of output tokens generated at previous time steps to generate a first decoder subnetwork output;
- for each subsequent decoder subnetwork in the sequence of decoder subnetworks:
- processing, using the subsequent decoder subnetwork, a subsequent decoder subnetwork input generated from i) the embedding of the input sequence and ii) the decoder subnetwork output generated by the previous decoder subnetwork in the sequence of decoder subnetworks to generate a subsequent decoder subnetwork output;
- and
- generating a new output element in the output sequence using the decoder subnetwork outputs.
Dependencies are inferred from the claim text.