High-fidelity generative image compression

A system comprising one or more computers and one or more storage devices storing instructions that when executed by one or more computers cause the one or more computers to perform operations for training an encoder neural network configured to receive a data item and to process the data item in accordance with current values of a plurality of encoder network parameters to output a compressed representation of the data item, wherein the training comprises, receiving a plurality of training data items, and, for each training data item:

• processing the training data item using the encoder neural network to generate a latent representation of the training data item;
• generating a compressed representation of the training data item from the latent representation of the training data item;
• processing the compressed representation using a decoder neural network to generate a reconstruction of the training data item;
• processing the reconstruction of the training data item using a discriminator neural network to generate a discriminator network output that specifies a discriminator's classification of the reconstruction of the training data item;
• evaluating a first loss function that depends on (ii) a reconstruction term measuring a quality of the reconstruction, and (iii) a discriminator term measuring a difference between the discriminator's classification of the reconstruction of the training data item and a ground truth classification of the training data item;
• and
• determining an update to the current values of the encoder network parameters based on determining a gradient with respect to the encoder network parameters of the first loss function.

One or more non-transitory computer-readable storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations for training an encoder neural network configured to receive a data item and to process the data item in accordance with current values of a plurality of encoder network parameters to output a compressed representation of the data item, wherein the training comprises, receiving a plurality of training data items, and, for each training data item:

• processing the training data item using the encoder neural network to generate a latent representation of the training data item;
• processing the latent representation using a hyper-encoder neural network to determine a conditional entropy model;
• generating a compressed representation of the training data item including entropy encoding the latent representation of the training data item in accordance with the conditional entropy model;
• processing the compressed representation using a decoder neural network to generate a reconstruction of the training data item;
• processing the reconstruction of the training data item using a discriminator neural network to generate a discriminator network output that specifies a discriminator's classification of the reconstruction of the training data item;
• evaluating a first loss function that depends on (i) a first term measuring a compression rate of the conditional entropy model, (ii) a second term measuring a quality of the reconstruction, and (iii) a third term measuring a difference between the discriminator's classification of the reconstruction of the training data item and a ground truth classification of the training data item;
• and
• determining an update to the current values of the encoder network parameters based on determining a gradient with respect to the encoder network parameters of the first loss function.