Video inpainting via user-provided reference frame

A non-transitory computer-readable medium having program code of a video editing tool stored thereon, wherein the program code, when executed by one or more processing devices, configures the one or more processing devices to perform operations comprising:

• accessing a set of video frames comprising a first frame and a second frame having respective annotations identifying a target region to be modified, the target region including a first target pixel at a first location in the first frame and a second target pixel at a second location in the second frame;
• interpolating, by the video editing tool and from a motion of a boundary of the target region within the set of video frames, a target motion of target pixels within the target region across the set of video frames;
• computing, by the video editing tool, confidence values for the set of video frames;
• identifying, by the video editing tool, a frame among the set of video frames with a confidence value less than a threshold confidence;
• and
• outputting, by the video editing tool, a suggestion to use the identified frame as a reference frame;
• generating the reference frame by at least applying, to the identified frame, a user-specified modification to the target region;
• and
• updating, based on reference color data from the user-specified modification, color data of the target region in the set of video frames to correspond to the target motion as interpolated.

A computing system comprising:

• a processing device;
• and
• a non-transitory computer-readable medium communicatively coupled to the processing device and storing program code,
• wherein the processing device is configured for executing the program code and thereby performing operations comprising:
• accessing a set of video frames comprising a first frame and a second frame having respective annotations identifying a target region, the target region including a first target pixel at a first location in the first frame and a second target pixel at a second location in the second frame,
• identifying a target motion of the target region across the set of video frames,
• computing confidence values associated with the target motion for the set of video frames;
• identifying a frame among the set of video frames with a confidence value less than a threshold confidence;
• and
• outputting a suggestion to use the identified frame as a reference frame;
• inserting, responsive to user input, the reference frame into the set of video frames, the reference frame comprising the identified frame modified by a user-specified modification to the target region, and
• updating color data of the target region in the set of video frames to correspond to the target motion interpolated from a boundary motion, wherein updating the color data comprises:
• identifying reference color data of a pixel at a reference location in the reference frame,
• updating first color data of the first target pixel with the reference color data, and
• updating second color data of the second target pixel with the updated first color data.