Back to results
US 12,755,390 B2Granted
Enhanced multi-view background matting for video conferencing
AssigneeIntel Corporation
InventorNiloufar Pourian
Life of this patent
Estimating the term…
Abstract
This disclosure describes systems, methods, and devices related to video background matte generation. A method may include receiving, by a first neural network trained to generate alpha mattes and foreground multi-camera images, first inputs generated by a second neural network; receiving, by the first neural network, second inputs comprising grayscale images and depth maps of the multi-camera images; and generating, by the first neural network, based on the first inputs and the second inputs, multi-view alpha mattes and multi-view foreground estimates for the multi-camera images.
Claims20
A method comprising:
- providing first inputs and second inputs in parallel to a first neural network, the first inputs including a plurality of estimated alpha mattes generated by a second neural network for a corresponding plurality of camera views associated with multi-camera images, the second inputs including a plurality of grayscale images and a plurality of depth maps corresponding to the plurality of camera views associated with the multi-camera images;
- and
- training the first neural network based on the first inputs, the second inputs, and a loss function to generate a plurality of multi-view alpha mattes and a plurality of multi-view foreground image estimates for the multi-camera images, the loss function based on a first plurality of differences and a second plurality of differences, the first plurality of differences between ones of a plurality of ground-truth alpha mattes and corresponding ones of the plurality of multi-view alpha mattes, the second plurality of differences between ones of a plurality of ground-truth foreground images and corresponding ones of the plurality of multi-view foreground image estimates.
A non-transitory computer-readable medium comprising computer-executable instructions to cause at least one processor circuit to at least:
- provide first inputs and second inputs in parallel to a first neural network, the first inputs including a plurality of estimated alpha mattes generated by a second neural network for a corresponding plurality of camera views associated with multi-camera images, the second inputs including a plurality of grayscale images and a plurality of depth maps corresponding to the plurality of camera views associated with the multi-camera images;
- and
- train the first neural network based on the first inputs, the second inputs, and a loss function to generate a plurality of multi-view alpha mattes and a plurality of multi-view foreground image estimates for the multi-camera images, the loss function based on a first plurality of differences and a second plurality of differences, the first plurality of differences between ones of a plurality of ground-truth alpha mattes and corresponding ones of the plurality of multi-view alpha mattes, the second plurality of differences between ones of a plurality of ground-truth foreground images and corresponding ones of the plurality of multi-view foreground image estimates.
A device comprising:
- memory;
- instructions;
- and
- at least one processor circuit to be programmed based on the instructions to:
- provide first inputs and second inputs in parallel to first neural network, the first inputs including a plurality of estimated alpha mattes generated by a second neural network for a corresponding plurality of camera views associated with multi-camera images, the second inputs comprising including a plurality of grayscale images and a plurality of depth maps corresponding to the plurality of camera views associated with the multi-camera images;
- and
- train the first neural network based on the first inputs, the second inputs, and a loss function to generate a plurality of multi-view alpha mattes and a plurality of multi-view foreground image estimates for the multi-camera images, the loss function based on a first plurality of differences and a second plurality of differences, the first plurality of differences between ones of a plurality of ground-truth alpha mattes and corresponding ones of the plurality of multi-view alpha mattes, the second plurality of differences between ones of a plurality of ground-truth foreground images and corresponding ones of the plurality of multi-view foreground image estimates.
4 more elements
Dependencies are inferred from the claim text.