High-performance and low-complexity neural compression from a single image, video or audio data

A system comprising:

• one or more computers;
• and
• one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform a method of encoding input data, the input data comprising input data values corresponding to respective input data grid points of an input data grid, the method comprising:
• (a) optimizing an objective function by jointly optimizing parameters of a synthesis neural network, parameters of a decoder neural network, and a set of latent values, the latent values corresponding to respective latent grid points of each of a plurality of latent grids having different respective resolutions, wherein the optimizing comprises, for each of a plurality of optimization iterations, determining gradients of the objective function using the latent values and using the gradients to update one or more of:
• the parameters of the synthesis neural network, the parameters of the decoder neural network, and the latent values;
• (b) quantizing the optimized latent values;
• and
• (c) encoding the quantized latent values using a probability distribution for the latent values, the probability distribution being defined by the decoder neural network,
• wherein the objective function comprises a reconstruction loss term comparing the input data values with corresponding reconstructed data values determined from the set of latent values using the synthesis neural network, and a compressibility term dependent on a probability of the set of latent values determined from the probability distribution defined by the decoder neural network, and
• wherein the optimizing further comprises updating the latent values by applying a soft-rounding function to each of the latent values.