De-noising images using machine learning
AssigneePixar
InventorsMark Meyer, Anthony DeRose, Steve Bako
Life of this patent
Estimating the term…
Abstract
The present disclosure relates to using a neural network to efficiently denoise images that were generated by a ray tracer. The neural network can be trained using noisy images generated with noisy samples and corresponding denoised or high-sampled images (e.g., many random samples). An input feature to the neural network can include color from pixels of an image. Other input features to the neural network, which would not be known in normal image processing, can include shading normal, depth, albedo, and other characteristics available from a computer-generated scene. After the neural network is trained, a noisy image that the neural network has not seen before can have noise removed without needing manual intervention.
Claims20
A method for reducing noise in a two-dimensional image obtained from a computer-generated scene, the method comprising:
- receiving a plurality of noisy subimages, wherein a subimage is at least a portion of an image, and wherein each of the plurality of noisy subimages is rendered using a respective set of rays in a corresponding computer-generated scene;
- receiving a plurality of refined subimages, each refined subimage corresponding to a noisy subimage of the plurality of noisy subimages and generated using a corresponding computer-generated scene, wherein a refined subimage has reduced noise relative to a corresponding noisy subimage;
- extracting values of a first set of features from pixels in each of the plurality of noisy subimages;
- determining values of a second set of features from information obtained from the corresponding computer-generated scenes from which the plurality of noisy subimages were generated;
- training a neural network based on the values of the features and the plurality of refined subimages, wherein the neural network outputs pixel values of an output subimage or outputs gradients of pixels of the output subimage, and wherein the training includes:
- generating, using the neural network, an output subimage for a noisy subimage;
- comparing the output subimage to a refined subimage that corresponds to the noisy subimage;
- and
- modifying one or more parameters of the neural network based on the comparison;
- receiving a new two-dimensional image rendered from a computer-generated scene;
- and
- generating, using the neural network after training, a new output image corresponding to the new two-dimensional image.
A non-transitory computer-readable medium storing a computer program that when executed by a processor of a computer cause the processor to implement a computer-implemented method comprising:
- receiving a plurality of noisy subimages, wherein a subimage is at least a portion of an image, and wherein each of the plurality of noisy subimages is rendered using a respective set of rays in a corresponding computer-generated scene;
- receiving a plurality of refined subimages, each refined subimage corresponding to a noisy subimage of the plurality of noisy subimages and generated using a corresponding computer-generated scene, wherein a refined subimage has reduced noise relative to a corresponding noisy subimage;
- extracting values of a first set of features from pixels in each of the plurality of noisy subimages;
- determining values of a second set of features from information obtained from the corresponding computer-generated scenes from which the plurality of noisy subimages were generated;
- training a neural network based on the values of the features and the plurality of refined subimages, wherein the neural network outputs pixel values of an output subimage or outputs gradients of pixels of the output subimage, and wherein the training includes:
- generating, using the neural network, an output subimage for a noisy subimage;
- comparing the output subimage to a refined subimage that corresponds to the noisy subimage;
- and
- modifying one or more parameters of the neural network based on the comparison;
- receiving a new two-dimensional image rendered from a computer-generated scene;
- and
- generating, using the neural network after training, a new output image corresponding to the new two-dimensional image.
A computing system comprising:
- one or more processors configured to:
- receive a plurality of noisy subimages, wherein a subimage is at least a portion of an image, and wherein each of the plurality of noisy subimages is rendered using a respective set of rays in a corresponding computer-generated scene;
- receive a plurality of refined subimages, each refined subimage corresponding to a noisy subimage of the plurality of noisy subimages and generated using a corresponding computer-generated scene, wherein a refined subimage has reduced noise relative to a corresponding noisy subimage;
- extract values of a first set of features from pixels in each of the plurality of noisy subimages;
- determine values of a second set of features from information obtained from the corresponding computer-generated scenes from which the plurality of noisy subimages were generated;
- train a neural network based on the values of the features and the plurality of refined subimages, wherein the neural network outputs pixel values of an output subimage or outputs gradients of pixels of the output subimage, and wherein the training includes:
- generating, using the neural network, an output subimage for a noisy subimage;
- comparing the output subimage to a refined subimage that corresponds to the noisy subimage;
- and
- modifying one or more parameters of the neural network based on the comparison;
- receive a new two-dimensional image rendered from a computer-generated scene;
- and
- generate, using the neural network after training, a new output image corresponding to the new two-dimensional image.
Dependencies are inferred from the claim text.