Image encoding and decoding, video encoding and decoding: methods, systems and training methods

A computer-implemented method for lossy or lossless image or video decoding, the method including the steps of:

• (i) receiving a first bitstream and a second bitstream;
• (ii) decoding the first bitstream using an arithmetic decoder, using predetermined entropy parameters, to produce a quantize z hyperlatent representation;
• (iii) decoding the quantized z hyperlatent representation using a hyperdecoder trained neural network, to obtain a location entropy parameter μy, an entropy scale parameter σy, and a context matrix Ay of a y latent representation;
• (iv) decoding the second bitstream using the entropy scale parameter σy in an arithmetic decoder, to output quantised latent residuals;
• (v) processing the quantized latent residuals, the location entropy parameter μy and the context matrix Ay, using an implicit decoding solver, to obtain a quantized y latent representation;
• (vi) decoding the quantized y latent representation using a decoder trained neural network, to obtain a reconstructed image.

A computer implemented method of training an encoder neural network, a decoder neural network, a hyperencoder neural network, and a hyperdecoder neural network, and entropy parameters, the neural networks, and the entropy parameters, being for use in lossy image or video compression, transmission and decoding, the method including the steps of:

• (i) receiving an input training image;
• (ii) encoding the input training image using the encoder neural network, to produce a y latent representation;
• (iii) encoding the y latent representation using the hyperencoder neural network, to produce a z hyperlatent representation;
• (iv) quantizing the z hyperlatent representation using an entropy parameter of the entropy parameters to produce a quantized z hyperlatent representation;
• (v) entropy encoding the quantized z hyperlatent representation into a first bitstream, using the entropy parameters;
• (vi) processing the quantized z hyperlatent representation using the hyperdecoder neural network to obtain a location entropy parameter μy, an entropy scale parameter σy, and a context matrix Ay of the y latent representation;
• (vii) processing the y latent representation, the location entropy parameter μy and the context matrix Ay, using an implicit encoding solver, to obtain quantized latent residuals;
• (viii) entropy encoding the quantized latent residuals into a second bitstream, using the entropy scale parameter σy;
• (ix) decoding the first bitstream using an arithmetic decoder, using the entropy parameters, to produce a quantized z hyperlatent representation;
• (x) decoding the quantized z hyperlatent representation using the hyperdecoder neural network, to obtain a location entropy parameter μy, an entropy scale parameter σy, and a context matrix Ay of a y latent representation;
• (xi) decoding the second bitstream using the entropy scale parameter σy in an arithmetic decoder, to output quantised latent residuals;
• (xii) processing the quantized latent residuals, the location entropy parameter μy and the context matrix Ay, using an (e.g. implicit)(e.g. linear) decoding solver, to obtain a quantized y latent representation;
• (xiii) decoding the quantized y latent representation using the decoder neural network, to obtain a reconstructed image;
• (xiv) evaluating a loss function based on differences between the reconstructed image and the input training image, and a rate term;
• (xv) evaluating a gradient of the loss function;
• (xvi) back-propagating the gradient of the loss function through the decoder neural network, through the hyperdecoder neural network, through the hyperencoder neural network and through the encoder neural network, and using the entropy parameters, to update weights of the encoder, decoder, hyperencoder and hyperdecoder neural networks, and to update the entropy parameters;
• and
• (xvii) repeating steps (i) to (xvi) using a set of training images, to produce a trained encoder neural network, a trained decoder neural network, a trained hyperencoder neural network and a trained hyperdecoder neural network, and trained entropy parameters;
• and
• (xviii) storing the weights of the trained encoder neural network, the trained decoder neural network, the trained hyperencoder neural network and the trained hyperdecoder neural network, and storing the trained entropy parameters.