Pixel blending for synthesizing video frames with occlusion and watermark handling

A method comprising:

• obtaining input frames from a video;
• generating a forward flow from a first input frame to a second input frame and a backward flow from the second input frame to the first input frame;
• generating an occlusion map at an interpolated frame coordinate using the forward flow and the backward flow;
• generating a consistency map at the interpolated frame coordinate using the forward flow and the backward flow;
• performing blending using the occlusion map and the consistency map to generate an interpolated frame at the interpolated frame coordinate;
• detecting a static letter or a watermark in the first input frame and the second input frame;
• generating a static mask based on the detected static letter or watermark;
• and
• replacing interpolated pixels in the interpolated frame with pixels in one of the first and second input frames based on the static mask.

A non-transitory computer readable medium containing instructions that when executed cause at least one processor to:

• obtain input frames from a video;
• generate a forward flow from a first input frame to a second input frame and a backward flow from the second input frame to the first input frame;
• generate an occlusion map at an interpolated frame coordinate using the forward flow and the backward flow;
• generate a consistency map at the interpolated frame coordinate using the forward flow and the backward flow;
• and
• perform blending using the occlusion map and the consistency map to generate an interpolated frame at the interpolated frame coordinate;
• wherein the instructions that when executed cause the at least one processor to perform the blending comprise instructions that when executed cause the at least one processor to:
• compare the occlusion map and the consistency map to thresholds in order to generate a reliability mask;
• when the reliability mask indicates that a pixel location is reliable, blend a pixel at the pixel location in the first input frame and a pixel at the pixel location in the second input frame to generate an interpolated pixel at the pixel location;
• and
• when the reliability mask indicates that a pixel location is not reliable, generate the interpolated pixel at the pixel location using the pixel at the pixel location in the first input frame.