Deferred neural rendering for view extrapolation

A system comprising:

• an image generator implemented on a processor and configured to generate a plurality of novel images, the image generator receiving as input a plurality of neural features selected from a neural texture atlas, the image generator also receiving as input one or more position guides identifying position information for a plurality of input image pixels, wherein the one or more position guides includes normal information, the normal information identifying a normal vector for one or more pixels in the plurality of input image pixels;
• an image discriminator implemented on the processor and configured to evaluate the plurality of novel images to determine a plurality of optimization values, the image discriminator comparing each of the plurality of novel images with a respective one of a corresponding plurality of input images, each of the plurality of novel images being generated from a respective camera pose relative to an object identical to that of the respective one of the plurality of input images;
• and
• a storage device configured to store updated neural features, the updated neural features and the image generator being updated based on the optimization values, the updated neural features supporting the generation of a designated novel image from a designated camera pose different from any of the plurality of input images.

One or more non-transitory computer readable media having instructions stored thereon for performing a method, the method comprising:

• generating a plurality of novel images using an image generator implemented on a processor, the image generator receiving as input a plurality of neural features selected from a neural texture atlas, the image generator also receiving as input one or more position guides identifying position information for a plurality of input image pixels, wherein the one or more position guides includes normal information, the normal information identifying a normal vector for one or more pixels in the plurality of input image pixels;
• evaluating the plurality of novel images using an image discriminator implemented on the processor to determine a plurality of optimization values, the image discriminator comparing each of the plurality of novel images with a respective one of a corresponding plurality of input images, each of the plurality of novel images being generated from a respective camera pose relative to an object identical to that of the respective one of the plurality of input images;
• updating the image generator and the neural features based on the optimization values;
• and
• storing the updated neural features on a storage device, the updated neural features supporting the generation of a designated novel image from a designated camera pose different from any of the plurality of input images.