Video frame interpolation via feature pyramid flows

A method of generating an interpolated image, the method comprising:

• receiving a first image and a second image at a neural network;
• extracting image features from the first image and warping the image features from the first image using a first plurality of parameters;
• extracting image features from the second image and warping the image features from the second image using a second plurality of parameters;
• generating a first candidate intermediate frame based on the warped first features and the warped second features;
• obtaining first multi-scale features associated with the image features extracted from the first image and warping the first multi-scale features using the first plurality of parameters;
• obtaining second multi-scale features associated with the image features extracted from the second image and warping the second multi-scale features using the second plurality of parameters;
• generating a second candidate intermediate frame based on the warped first multi-scale features and the warped second multi-scale features;
• and
• blending the first candidate intermedia frame with the second candidate intermediate frame to generate the interpolated image.

A system comprising:

• a processor;
• and
• memory including instructions, which when executed by the processor, causes the processor to:
• obtain first features from a first image;
• obtain first multi-scale features from encoded first features;
• obtain second features from a second image;
• obtain second multi-scale features from encoded second features;
• warp the first features from the first image using a first plurality of parameters,
• warp the first multi-scale features using the first plurality of parameters;
• warp the second features from the second image using a second plurality of parameters,
• warp the second multi-scale features using the second plurality of parameters;
• generate a first candidate intermediate frame based on the warped first features and the warped second features;
• generate a second candidate intermediate frame based on the warped first multi-scale features and the warped second multi-scale features;
• and
• blend the first candidate intermediate frame and the second candidate intermediate frame to generate an interpolated frame.