Photo relighting using deep neural networks and confidence learning
AssigneeGoogle Inc.
InventorsTiancheng Sun, Yun-ta Tsai, Jonathan Barron
Life of this patent
Estimating the term…
Abstract
Apparatus and methods related to applying lighting models to images of objects are provided. A neural network can be trained to apply a lighting model to an input image. The training of the neural network can utilize confidence learning that is based on light predictions and prediction confidence values associated with lighting of the input image. A computing device can receive an input image of an object and data about a particular lighting model to be applied to the input image. The computing device can determine an output image of the object by using the trained neural network to apply the particular lighting model to the input image of the object.
Claims20
A computer-implemented method, comprising:
- training a neural network to apply a lighting model to an input image utilizing confidence learning that is based on light predictions and prediction confidence values associated with lighting of the input image, wherein the light predictions comprise a prediction of an original light model indicative of a location of one or more environmental light sources with reference to an object in the input image;
- receiving a particular input image of a particular object and data about a particular lighting model to be applied to the particular input image at a computing device;
- and
- determining, by the computing device and based on a particular original light model predicted by the neural network, an output image of the particular object by using the trained neural network to apply the particular lighting model to the particular input image of the particular object.
A computing device, comprising:
- one or more processors;
- and
- data storage, wherein the data storage has stored thereon computer-executable instructions that, when executed by the one or more processors, cause the computing device to carry out functions comprising:
- training a neural network to apply a lighting model to an input image utilizing confidence learning that is based on light predictions and prediction confidence values associated with lighting of the input image, wherein the light predictions comprise a prediction of an original light model indicative of a location of one or more environmental light sources with reference to an object in the input image;
- receiving a particular input image of a particular object and data about a particular lighting model to be applied to the particular input image at a computing device;
- and
- determining, by the computing device and based on a particular original light model predicted by the neural network, an output image of the particular object by using the trained neural network to apply the particular lighting model to the particular input image of the particular object.
An article of manufacture comprising one or more computer readable media having computer-readable instructions stored thereon that, when executed by one or more processors of a computing device, cause the computing device to carry out functions comprising:
- training a neural network to apply a lighting model to an input image utilizing confidence learning that is based on light predictions and prediction confidence values associated with lighting of the input image, wherein the light predictions comprise a prediction of an original light model indicative of a location of one or more environmental light sources with reference to an object in the input image;
- receiving a particular input image of a particular object and data about a particular lighting model to be applied to the particular input image at a computing device;
- and
- determining, by the computing device and based on a particular original light model predicted by the neural network, an output image of the particular object by using the trained neural network to apply the particular lighting model to the particular input image of the particular object.
Dependencies are inferred from the claim text.