3D modeling based on neural light field

A system comprising:

• at least one processor of a device;
• and
• a memory component having instructions stored thereon that, when executed by the at least one processor, cause the at least one processor to perform operations comprising:
• receiving a set of two-dimensional (2D) images representing a first view of a real-world environment;
• applying a machine learning model comprising a neural light field network to the set of 2D images to predict pixel values of a target image representing a second view of the real-world environment, the machine learning model being trained to map a ray origin and direction directly to a given pixel value, the pixel values of the target image being predicted without processing multiple points along a camera ray directed to the given pixel value;
• and
• generating a three-dimensional (3D) model of the real-world environment based on the set of 2D images and the predicted target image.

A non-transitory computer-readable storage medium having stored thereon instructions that, when executed by at least one processor of a device, cause the at least one processor to perform operations comprising:

• receiving a set of two-dimensional (2D) images representing a first view of a real-world environment;
• applying a machine learning model comprising a neural light field network to the set of 2D images to predict pixel values of a target image representing a second view of the real-world environment, the machine learning model being trained to map a ray origin and direction directly to a given pixel value, the pixel values of the target image being predicted without processing multiple points along a camera ray directed to the given pixel value;
• and
• generating a three-dimensional (3D) model of the real-world environment based on the set of 2D images and the predicted target image.