US 12,737,964 B2 · Increasing levels of detail for neural fields using diffusion models

Abstract

Systems and methods of the present disclosure include providing higher levels of detail (LODs) for generated three-dimensional (3D) models, such as those represented by neural radiance fields (NeRFs). A 3D model may be presented to a user in which the user may request additional LODs, such as to zoom into the image or to receive information about features within the image. A request to generate finer levels of detail may include using one or more diffusion models to generate images at higher resolutions and/or to hallucinate finer details based on information extracted from the original image or text prompts. Newly generated images may then be added to a set of images associated with the 3D models to enable later model generation to have finer details.

Claims20

Dependent claims

A computer-implemented method, comprising:

determining a target level of detail for a three-dimensional (3D) volume;
generating, using an image generation network and based at least on a current view representing the 3D volume, an updated view representing a selected portion of the 3D volume at the target level of detail, wherein at least a portion of the 3D volume is removed in the updated view;
providing, responsive to the target level of detail, the updated view;
adding the updated view to a set of images associated with the 3D volume;
and
updating the 3D volume, based at least on the updated view.

3 more elements

A system, comprising:

one or more processors comprising processing circuitry to generate an output image with a finer level of detail (LOD) than an input image generated using a neural radiance field (NeRF), to update the NeRF using a set of images that includes the output image, and to render the output image at the finer LOD with at least a portion of the input image removed.

Dependencies are inferred from the claim text.

Increasing levels of detail for neural fields using diffusion models

Life of this patent

Increasing levels of detail for neural fields using diffusion models

Life of this patent

Abstract

Claims20