Real-time neural network radiance caching for path tracing

A system, comprising:

• a memory storing data for a scene;
• a processor that is connected to the memory and configured to generate images of the scene by:
• processing, by a neural network radiance cache model, a three-dimensional (3D) position associated with a light transport path through a scene to produce a radiance prediction at the 3D position;
• generating an image of the scene corresponding to at least one camera using the radiance prediction;
• generating an extended light transport path ending at a terminal 3D position;
• processing the terminal 3D position to produce a second radiance prediction at the terminal 3D position;
• and
• updating weights of the neural network radiance cache model based on the second radiance prediction.

A non-transitory computer-readable media storing computer instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of:

• processing, by a neural network radiance cache model, a three-dimensional (3D) position associated with a light transport path through a scene to produce a radiance prediction at the 3D position;
• generating an image of the scene corresponding to at least one camera using the radiance prediction;
• generating an extended light transport path ending at a terminal 3D position;
• processing the terminal 3D position to produce a second reflected radiance at the terminal 3D position;
• and
• updating weights of the neural network radiance cache model based on the second radiance prediction.