Generalized mask propagation for video segmentation

A non-transitory computer-readable medium storing executable instructions, which when executed by a processing device, cause the processing device to perform operations comprising:

• determining an image segmentation mask which masks an object of a frame of a video sequence using the frame and an image segmentation module of a segmentation system;
• determining a mask propagation mask which masks the object of the frame of the video sequence using the frame, a representation of a previous frame of the video sequence, and a mask propagation module of the segmentation system;
• determining a frame mask which masks the object of the frame of the video sequence based on a comparison of the image segmentation mask and the mask propagation mask;
• and
• displaying the frame mask of the video sequence.

A system comprising:

• a memory component;
• and
• a processing device coupled to the memory component, the processing device to perform operations comprising:
• determining a first mask which masks each object of a plurality of objects of a frame of a video sequence using the frame and an image segmentation module;
• determining a second mask which masks each object of the plurality of objects of the frame using the frame, a representation of a previous frame of the video sequence, and a mask propagation module;
• comparing the first mask and the second mask to obtain a coarse estimation of a third mask;
• refining the coarse estimation of the third mask to obtain a refined estimation using the frame, the coarse estimation of the third mask, the representation of the previous frame of the video sequence, and the mask propagation module.