Video super-resolution using deep neural networks

A method comprising:

• obtaining an input video sequence comprising a plurality of input video frames having an input resolution, wherein the input resolution is dynamically reduced based on a strength of a connection with a destination user device;
• and
• processing the input video sequence using a neural network to generate an output video sequence, comprising, for each of the plurality of input video frames:
• processing the input video frame using the neural network to generate an output video frame having a target output resolution that is higher than the input resolution, wherein a resolution of the plurality of input video frames is incrementally upsampled to the target output resolution,
• wherein the neural network has been trained by performing operations comprising:
• obtaining a training input video frame;
• processing the training input video frame using a trained machine learning model that is configured detect faces in images;
• processing, using an output of the trained machine learning model, the training input video frame to generate an updated training input video frame that isolates a detected face in the training input video frame;
• processing the updated training input video frame using the neural network to generate a predicted output video frame;
• and
• updating a plurality of parameters of the neural network according to an error of the predicted output video frame.

A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one more computers to perform operations comprising:

• obtaining an input video sequence comprising a plurality of input video frames;
• determining i) an input resolution of the plurality of input video frames and ii) a target output resolution of the plurality of input video frames, wherein the target output resolution is higher than the input resolution, wherein the input resolution is dynamically reduced based on a strength of a connection with a destination user device;
• and
• processing the input video sequence using a neural network to generate an output video sequence, comprising, for each of the plurality of input video frames:
• processing the input video frame using the neural network to generate an output video frame having the target output resolution, comprising processing the input video frame using a subnetwork of the neural network corresponding to the input resolution of the plurality of input video frames, wherein a resolution of the plurality of input video frames is incrementally upsampled to the target output resolution,
• wherein the neural network has been configured through training to process input video frames having one of a predetermined set of possible input resolutions and to generate output video frames having one of a predetermined set of possible output resolutions.