Occlusion processing for frame rate conversion using deep learning

An apparatus for converting a frame rate of an input video, the apparatus comprising:

• a memory storing instructions;
• and
• a processor configured to execute the instructions to:
• perform a motion estimation by generating at least one motion field for at least one pair of frames of the input video, wherein the at least one motion field comprises a set of motion vectors for each block of a first reference frame of the input video, the set of motion vectors indicating to a second reference frame of the input video;
• prepare first data for a predetermined interpolation phase for each block of an interpolated frame, wherein the first data comprise at least one parameter for a pre-trained occlusion processing (OcC) convolutional neural network (CNN), the at least one parameter being obtained based on the at least one motion field;
• perform occlusion correction with the OcC CNN for each block of the interpolated frame by predicting weights associated with the set of motion vectors based on the prepared first data;
• and
• perform motion compensation for each block or for each pixel of the interpolated frame by processing data from the OcC CNN.