US 10,255,681 B2 · Image matting using deep learning

Abstract

Methods and systems are provided for generating mattes for input images. A neural network system can be trained where the training includes training a first neural network that generates mattes for input images where the input images are synthetic composite images. Such a neural network system can further be trained where the training includes training a second neural network that generates refined mattes from the mattes produced by the first neural network. Such a trained neural network system can be used to input an image and trimap pair for which the trained system will output a matte. Such a matte can be used to extract an object from the input image. Upon extracting the object, a user can manipulate the object, for example, to composite the object onto a new background.

Claims20

Dependent claims

A computer-implemented method for training a neural network system to generate mattes for images, the method comprising:

inputting, into a first neural network, a training image and trimap pair, the trimap indicating a blended region in which one or more pixels include an unknown combination of foreground and background colors;
determining, by the first neural network, a percentage of foreground color for each of the one or more pixels in the blended region, the first neural network using color, structure, and texture information to determine percentages of foreground color in association with the one or more pixels;
generating, by the first neural network, a training matte for the training image using the percentages of foreground color for each of the one or more pixels in the blended region;
identifying, by the first neural network, error associated with the training matte using a loss function;
and
adjusting the first neural network based on the identified error associated with the training matte.

3 more elements

One or more non-transitory computer-readable media having a plurality of executable instructions embodied thereon, which, when executed by one or more processors, cause the one or more processors to perform a method for generating mattes for input images, the method comprising:

receiving an input image and trimap pair, the trimap indicating a blended region in which one or more pixels include an unknown combination of foreground and background colors;
determining a percentage of foreground color for each of the one or more pixels in the blended region via a neural network system that utilizes color, structure, and texture information to determine percentages of foreground color in association with the one or more pixels, wherein the neural network system comprises a first neural network trained to generate mattes based on adjusting for errors determined using color, structure, and texture information;
and
generating, via the neural network system, a matte for the input image using the percentages of foreground color for each of the one or more pixels in the blended region.

1 more element

Dependencies are inferred from the claim text.

Image matting using deep learning

Life of this patent

Image matting using deep learning

Life of this patent

Abstract

Claims20