Video frame segmentation using reduced resolution neural network and masks from previous frames
AssigneeIntel Corporation
InventorsAmir Goren, Noam Elron, Noam Levy
Life of this patent
Estimating the term…
Abstract
An example apparatus for video frame segmentation includes a receiver to receive a current video frame to be segmented. The apparatus also includes a segmenting neural network to receive a previous mask including a segmentation mask corresponding to a previous frame and generate a segmentation mask for the current frame based on the previous mask and the video frame.
Claims22
An apparatus for video frame segmentation, comprising:
- receiver circuitry to receive a current video frame to be segmented;
- downscaler circuitry to reduce a resolution of the current video frame to generate a reduced resolution frame;
- a segmenting neural network to:
- receive a previous mask including a first reduced resolution segmentation mask corresponding to a previous frame;
- and
- generate a second reduced resolution segmentation mask for the current video frame, the second reduced resolution segmentation mask based on the previous mask and the reduced resolution video frame;
- mask feedback circuitry to send the second reduced resolution segmentation mask to the segmenting neural network after a delay, the second reduced resolution segmentation mask to be used as the previous mask for a subsequent reduced resolution video frame, the delay corresponding to a video frame duration;
- and
- an upscaling neural network trained to upscale the second reduced resolution segmentation mask into a resolution of the current video frame.
At least one computer readable medium for segmenting video having instructions stored therein that, in response to being executed on a computing device, cause the computing device to:
- access a current video frame to be segmented;
- reduce a resolution of the current video frame to generate a reduced resolution frame;
- access a previous mask including a first reduced resolution segmentation mask corresponding to a previous frame;
- generate, via a segmenting neural network, a second reduced resolution segmentation mask for the current video frame, the second reduced resolution segmentation mask based on the previous mask and the reduced resolution video frame;
- send the second reduced resolution segmentation mask to the segmenting neural network after a delay, the second reduced resolution segmentation mask to be used as the previous mask for a subsequent reduced resolution video frame, the delay corresponding to a video frame duration;
- upscale, via an upscaling neural network, the second reduced resolution segmentation mask into a resolution of the current video frame;
- and
- modify the current video frame based on the upscaled second segmentation mask.
An apparatus for video frame segmentation, comprising:
- memory;
- instructions;
- and
- processor circuitry to execute the instructions to:
- receive a current video frame to be segmented;
- reduce a resolution of the current video frame to generate a reduced resolution frame;
- receive a previous mask including a first reduced resolution segmentation mask corresponding to a previous frame;
- generate, via a segmenting neural network, a second reduced resolution segmentation mask for the current video frame, the second reduced resolution segmentation mask based on the previous mask and the reduced resolution video frame;
- send the second reduced resolution segmentation mask to the segmenting neural network after a delay, the second reduced resolution segmentation mask to be used as the previous mask for a subsequent reduced resolution video frame, the delay corresponding to a video frame duration;
- and
- upscale, via an upscaling neural network, the second reduced resolution segmentation mask into a resolution of the current video frame.
Dependencies are inferred from the claim text.