Methods and systems for training quantized neural radiance field

A device comprising:

• a processor;
• and
• a memory storing instructions executable by the processor, wherein the instructions are executed by the processor, the instructions cause the processor to perform operations, comprising:
• encoding a radiance field of an object onto a machine learning model:
• conducting, based on a set of training images of the object, a training process on the machine learning model to obtain a trained machine learning model, wherein the training process comprises a first training process using a plurality of first test sample points followed by a second training process using a plurality of second test sample points, each of the plurality of first test sample points and the plurality of second test sample points located on a plurality of training rays derived from the training images, the plurality of second test sample points located within a threshold distance from a surface region of the object;
• obtaining target view parameters indicating a viewpoint and a view direction of the object;
• obtaining, based on the target view parameters, a plurality of sample rays associated with a target image of the object, the target image associated with the viewpoint and the view direction;
• obtaining render sample points on the plurality of sample rays;
• and
• rendering, by inputting the render sample points to the trained machine learning model, colors associated with pixels of the target image.

A non-transitory storage medium of storing instructions executable by a process, wherein, the instructions are executed by the processors, the instructions cause the processor to perform operations, comprising:

• encoding a radiance field of an object onto a machine learning model;
• conducting, based on a set of training images of the object, a training process on the machine learning model to obtain a trained machine learning model, wherein the training process comprises a first training process using a plurality of first test sample points followed by a second training process using a plurality of second test sample points, each of the plurality of first test sample points and the plurality of second test sample points located on a plurality of training rays derived from the training images, the plurality of second test sample points located within a threshold distance from a surface region of the object;
• obtaining target view parameters indicating a viewpoint and a view direction of the object;
• obtaining, based on the target view parameters, a plurality of sample rays associated with a target image of the object, the target image associated with the viewpoint and the view direction;
• obtaining render sample points on the plurality of sample rays associated with the target image;
• and
• rendering, by inputting the render sample points to the trained machine learning model, colors associated with pixels of the target image.