Neural local attention modules for denoising deep monte carlo renderings

A method for training a machine learning model to denoise deep images comprising pluralities of pixels, each pixel corresponding to one or more bins, each deep image thereby comprising a plurality of bins, wherein the machine learning model comprises an embedding sub-model and a denoising sub-model, and wherein the method is performed by a computer system and comprises performing an iterative training process until a terminating condition has been met, the method comprising:

• sampling a batch of training deep images comprising one or more training deep images, each training deep image comprising a plurality of training bins;
• generating, using the embedding sub-model, one or more training deep image embeddings by performing, for each training deep image of the one or more training deep images:
• determining a plurality of local bin sets corresponding to the plurality of training bins, each local bin set comprising a plurality of local bins from the plurality of training bins and a respective focal training bin, each plurality of local bins being within a specified distance of the respective focal training bin,
• generating, for each focal training bin using the embedding sub-model, a training bin embedding based on attention of a corresponding local bin set, thereby generating a plurality of training bin embeddings, and
• generating a training deep image embedding based on the plurality of training bin embeddings, thereby generating the one or more training deep image embeddings;
• generating one or more denoised training deep images by generating, for each training deep image, a denoised training deep image by generating a plurality of denoised training bins using the denoising sub-model applied to the plurality of training bins of a corresponding training deep image and a corresponding training deep image embedding, thereby generating the one or more denoised training deep images;
• determining one or more loss values based on the one or more denoised training deep images;
• updating a parameter set of the machine learning model based on the one or more loss values, thereby training the machine learning model;
• and
• if the terminating condition has not been met, repeating the iterative training process until the terminating condition has been met, otherwise completing the iterative training process.

A computer system comprising:

• one or more processors;
• and
• a non-transitory computer readable medium coupled to the one or more processors, the non-transitory computer readable medium comprising code executable by the one or more processors for performing a method for denoising a deep image comprising a plurality of pixels, each pixel corresponding to one or more bins, the deep image thereby comprising a plurality of bins, the method comprising:
• generating, using an embedding sub-model, a deep image embedding by:
• determining a plurality of local bin sets corresponding to the plurality of bins, each local bin set comprising a plurality of local bins from the plurality of bins and a respective focal bin, each plurality of local bins being within a specified distance of the respective focal bin,
• generating, for each focal bin using the embedding sub-model, a bin embedding based on attention of a corresponding local bin set, thereby generating a plurality of bin embeddings,
• generating the deep image embedding based on the plurality of bin embeddings;
• and
• generating a denoised deep image by generating a plurality of denoised bins using a denoising sub-model applied to the plurality of bins of the deep image and the deep image embedding, wherein the denoised deep image comprises the plurality of denoised bins.