Kernel-predicting convolutional neural networks for denoising

A method of denoising images rendered by Monte Carlo (MC) path-tracing, the method comprising:

• receiving a plurality of input images, each input image having a first number of pixels and including input image data for each respective pixel obtained by MC path-tracing, the input image data comprising color data for each respective pixel;
• receiving a plurality of reference images, each reference image corresponding to a respective input image and having a second number of pixels, each reference image including reference image data for each respective pixel;
• training a neural network using the plurality of input images and the plurality of reference images, the neural network including:
• an input layer having a first number of input nodes for receiving input image data for each respective pixel of a respective input image;
• a plurality of hidden layers, each hidden layer having a respective number of nodes, each node of a respective hidden layer processing data of a plurality of nodes of a preceding hidden layer using a plurality of parameters associated with the plurality of nodes, with each node of a first hidden layer of the plurality of hidden layers processing data of a plurality of nodes of the input layer;
• an output layer having a second number of output nodes associated with a neighborhood of pixels around each pixel of the input image, each node of the output layer processing data of a plurality of nodes of a last hidden layer of the plurality of hidden layers to obtain a respective output value;
• and
• a reconstruction module coupled to the output layer for generating a respective output image corresponding to the respective input image, the respective output image having the second number of pixels, each respective pixel having color data relating to a weighted combination of the color data for the neighborhood of pixels around a corresponding pixel of the input image using the output values associated with the second number of output nodes as weights;
• wherein training the neural network includes, for each respective input image, optimizing the plurality of parameters associated with the plurality of nodes of each hidden layer by comparing the respective output image to a corresponding reference image to obtain a plurality of optimized parameters;
• receiving a new input image obtained by MC path-tracing;
• and
• generating a new output image corresponding to the new input image by passing the new input image through the neural network using the plurality of optimized parameters, the new output image being less noisy than the new input image.

A method of denoising images rendered by Monte Carlo (MC) path-tracing, the method comprising:

• receiving a plurality of input images, each input image having a first number of pixels and including a diffuse buffer and a specular buffer, the diffuse buffer including diffuse input image data for each respective pixel, the specular buffer including specular input image data for each respective pixel, the diffuse input image data comprising diffuse color data for each respective pixel, and the specular input image data comprising specular color data for each respective pixel;
• receiving a plurality of reference images, each reference image corresponding to a respective input image and having a second number of pixels, each reference image including a diffuse buffer and a specular buffer, the diffuse buffer including diffuse reference image data for each respective pixel, the specular buffer including specular reference image data for each respective pixel;
• training a first neural network using the diffuse buffers of the plurality of input images and the diffuse buffers of the plurality of reference images, the first neural network including:
• a diffuse input layer for receiving diffuse input image data for each respective pixel of a respective input image;
• a plurality of diffuse hidden layers, each diffuse hidden layer including a plurality of nodes, each node of a respective diffuse hidden layer processing data of a plurality of nodes of a preceding diffuse hidden layer using a plurality of first parameters associated with the plurality of nodes, with each node of a first diffuse hidden layer of the plurality of diffuse hidden layers processing data of a plurality of nodes of the diffuse input layer;
• a diffuse output layer having a first number of output nodes associated with a first neighborhood of pixels around each pixel of the input image, each node of the diffuse output layer processing data of a plurality of nodes of a last diffuse hidden layer of the plurality of hidden layers to obtain a respective diffuse output value;
• and
• a diffuse reconstruction module coupled to the diffuse output layer for generating a respective diffuse output image corresponding to the respective input image, the respective diffuse output image having the second number of pixels, each respective pixel having diffuse color data relating to a weighted combination of the diffuse color data for the first neighborhood of pixels around a corresponding pixel of the input image using the diffuse output values associated with the first number of output nodes as weights;
• wherein training the first neural network includes, for each respective input image, optimizing the plurality of first parameters associated with the plurality of nodes of each diffuse hidden layer by comparing the respective diffuse output image to the diffuse buffer of a corresponding reference image to obtain a plurality of optimized first parameters;
• training a second neural network using the specular buffers of the plurality of input images and the specular buffers of the plurality of reference images, the second neural network including:
• a specular input layer for receiving specular input image data for each respective pixel of a respective input image;
• a plurality of specular hidden layers, each specular hidden layer including a plurality of nodes, each node of a respective specular hidden layer processing data of a plurality of nodes of a preceding specular hidden layer using a plurality of second parameters associated with the plurality of nodes, with each node of a first specular hidden layer of the plurality of specular hidden layers processing data of a plurality of nodes of the specular input layer;
• a specular output layer having a second number of output nodes associated with a second neighborhood of pixels around each pixel of the input image, each node of the specular output layer processing data of a plurality of nodes of a last specular hidden layer of the plurality of specular hidden layers to obtain a respective specular output value;
• and
• a specular reconstruction module coupled to the specular output layer for generating a respective specular output image corresponding to the respective input image, the respective specular output image having the second number of pixels, each respective pixel having specular color data relating to a weighted combination of the specular color data for the second neighborhood of pixels around a corresponding pixel of the input image using the specular output values associated with the second number of output nodes as weights;
• wherein training the second neural network includes, for each respective input image, optimizing the plurality of second parameters associated with the plurality of nodes of each specular hidden layer by comparing the respective specular output image to the specular buffer of a corresponding reference image to obtain a plurality of optimized second parameters;
• receiving a new input image obtained by MC path-tracing, the new input image including a diffuse buffer and a specular buffer;
• generating a new diffuse output image corresponding to the new input image by passing the diffuse buffer of the new input image through the first neural network using the plurality of optimized first parameters;
• generating a new specular output image corresponding to the new input image by passing the specular buffer of the new input image through the second neural network using the plurality of optimized second parameters;
• and
• generating a new output image by combining the new diffuse output image and the new specular output image, the new output image being less noisy than the new input image.