Image compression and decoding, video compression and decoding: methods and systems

A system for lossy image or video compression, transmission and decoding, the system including a first computer system, a first trained neural network, a second computer system, a second trained neural network, a third trained neural network, a fourth trained neural network and a trained neural network identical to the fourth trained neural network, wherein:

• (i) the first computer system is configured to receive an input image;
• (ii) the first computer system is configured to encode the input image using a first trained neural network, to produce a y latent representation;
• (iii) the first computer system is configured to encode they latent using a second trained neural network, to produce a z latent representation;
• (iv) the first computer system is configured to encode the z latent using a third trained neural network, to produce a w latent representation;
• (v) the first computer system is configured to entropy encode the w latent into a first bitstream;
• (vi) the first computer system is configured to entropy encode the z latent into a second bitstream;
• (vii) the first computer system is configured to entropy encode the y latent into a third bitstream;
• (viii) the first computer system is configured to transmit the first bitstream, the second bitstream, and the third bitstream to the second computer system;
• (ix) the second computer system is configured to entropy decode the first bitstream to produce the w latent;
• (x) the second computer system is configured to process the w latent using a fourth trained neural network;
• (xi) the second computer system is configured to entropy decode the second bitstream using the processed w latent to produce the z latent;
• (xii) the second computer system is configured to process the z latent using a fifth trained neural network;
• (xiii) the second computer system is configured to entropy decode the third bitstream using the processed z latent to produce the y latent;
• and
• (xiv) the second computer system is configured to use a sixth trained neural network to produce an output image from the y latent, wherein the output image is an approximation of the input image.

A computer implemented method of training a first neural network, a second neural network, a third neural network, a fourth neural network, a fifth neural network, and a sixth neural network, the neural networks being for use in lossy image or video compression, transmission and decoding, the method including the steps of:

• (i) receiving an input training image;
• (ii) encoding the input training image using the first neural network, to produce a y latent representation;
• (iii) encoding they latent using the second neural network, to produce a z latent representation;
• (iv) encoding the z latent representation using the third neural network, to produce a w latent representation;
• (v) entropy encoding the w latent into a first bitstream;
• (vi) entropy encoding the z latent into a second bitstream;
• (vii) entropy encoding the y latent into a third bitstream;
• (viii) processing the w latent using the fourth neural network;
• (ix) using the processed w latent, together with the second bitstream, to obtain the z latent;
• (x) processing the z latent using the fifth neural network;
• (xii) using the processed z latent, together with the first bitstream, to obtain the y latent;
• (xiii) using the sixth neural network to produce an output image from the y latent, wherein the output image is an approximation of the input training image;
• (xiv) evaluating a loss function based on differences between the output image and the input training image;
• (xv) evaluating a gradient of the loss function;
• (xvi) back-propagating the gradient of the loss function through the sixth, fifth, fourth, third, second and first t neural networks, to update weights of the first, second, third, fourth, fifth, and sixth neural networks;
• and
• (xvii) repeating steps (i) to (xvi) using a set of training images, to produce a trained first, second, third, fourth, fifth, and sixth neural networks, and
• (xviii) storing the weights of the trained first, second, third, fourth, fifth, and sixth neural networks.