Video inpainting via confidence-weighted motion estimation

A non-transitory computer-readable medium having program code stored thereon that, when executed by one or more processing devices, causes the one or more processing devices to perform operations comprising:

• accessing a video having a target region;
• a step for updating video content in the target region based on a confidence-weighted motion estimation for the target region, wherein the step for updating video content comprises:
• computing a forward optical flow and a backward optical flow of a particular boundary pixel of the target region and with respect to a first video frame and a second video frame;
• and
• applying a sigmoidal function to a difference between the forward optical flow and the backward optical flow, wherein a particular confidence value includes or is derived from an output of the sigmoidal function,
• wherein an increase in the difference between the forward optical flow and the backward optical flow corresponds to a decrease in the particular confidence value;
• and
• presenting the updated video content on a display device.

A system comprising:

• a processing device;
• and
• a non-transitory computer-readable medium communicatively coupled to the processing device,
• wherein the processing device is configured to execute program code stored on the non-transitory computer-readable medium and thereby perform operations comprising:
• accessing a scene comprising a first video frame and a second video frame, the accessed scene having an annotation identifying a target region to be modified in one or more video frames comprising the scene,
• computing a boundary motion for a boundary of the target region within the scene, wherein the boundary includes boundary pixels neighboring the target region in the first video frame and the second video frame,
• assigning confidence values to the boundary pixels, respectively, wherein assigning a particular confidence value comprises:
• computing a forward optical flow and a backward optical flow of a particular boundary pixel with respect to the first video frame and the second video frame;
• and
• applying a sigmoidal function to a difference between the forward optical flow and the backward optical flow, wherein the particular confidence value includes or is derived from an output of the sigmoidal function, and
• wherein an increase in the difference between the forward optical flow and the backward optical flow corresponds to a decrease in the particular confidence value,
• interpolating a target motion of a target pixel within the target region from the boundary motion, wherein the particular confidence value of the particular boundary pixel controls a contribution of a particular motion of the particular boundary pixel to the target motion as interpolated from the boundary motion, and
• updating color data of the target pixel to correspond to the target motion interpolated from the boundary motion.