Relightable 3D Reconstruction and View Synthesis
AssigneeGoogle Inc.
InventorsPhilipp Henzler, Ricardo Martin-Brualla, Xiaoming Zhao, Pratul Preeti Srinivasan, Dor Verbin, Keunhong Park
Life of this patent
Estimating the term…
Loading patent…
AssigneeGoogle Inc.
InventorsPhilipp Henzler, Ricardo Martin-Brualla, Xiaoming Zhao, Pratul Preeti Srinivasan, Dor Verbin, Keunhong Park
Estimating the term…
Provided are systems and methods for relightable view synthesis that can process a set of source images captured under unknown lighting conditions to produce 3D reconstructions under novel target lighting and from novel viewpoints or poses. Initially, an example method includes obtaining source images and target lighting data, followed by generating radiance data using a source neural scene representation and a rendering engine. A machine-learned relighting diffusion model can then be employed to process the source images and radiance data to generate re-lit images. These images are subsequently used to train a latent neural radiance field model, which, upon querying following training, can generate synthetic images from novel poses under the target lighting. The proposed technology can be beneficial for applications in virtual reality, filmmaking, game development, and other settings, offering a robust alternative to traditional inverse rendering methods by leveraging advanced machine learning techniques to handle complex lighting scenarios.
A computer-implemented method for performing relightable view synthesis, the method comprising:
A computing system comprising one or more processors and one or more non-transitory computer-readable media that collectively store instructions that when executed by the one or more processors cause the computing system to perform operations, the operations comprising:
One or more non-transitory computer-readable media that store a latent neural radiance field model configured to generate a synthetic image that depicts the scene with the target lighting from a novel pose, wherein the latent neural radiance field model has previously been trained by performance of training operations, the training operations comprising:
Dependencies are inferred from the claim text.