Method and data processing system for lossy image or video encoding, transmission and decoding

A method of training one or more neural networks, the one or more neural networks being for use in lossy video encoding, transmission and decoding, the method comprising the steps of:

• receiving a first frame and a second frame at a first computer system;
• determining a first flow between the first frame and the second frame;
• determining a second flow based on the first frame and the second frame;
• encoding an input based on the first flow and the second flow using a first neural network to produce a latent representation;
• transmitting the latent representation to a second computer system;
• decoding the latent representation using a second neural network to produce an output flow;
• using the output flow to obtain an output frame, wherein the output frame is an approximation of the second frame;
• evaluating a function based on a difference between the output frame and the second frame;
• updating the parameters of the first neural network and the second neural network based on the evaluated function;
• and
• repeating the above steps using a first set of input frames to produce a first trained neural network and a second trained neural network;
• wherein each of the first flow and the second flow define a transformation of a part of an image from a first position in the first frame to a second position in the second frame.