# US 2025/0166288 A1 · Increasing levels of detail for neural fields using diffusion models

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0166288 A1
- Document ID: US20250166288A1
- Status: Published application
- Kind: A1
- Application number: 18/513,105
- Filed: 2023-11-17
- Published: 2025-05-22
- Earliest claimed: 2023-11-17
- Assignees: NVIDIA Corporation
- Applicants: Nvidia Corporation
- Inventors: Or Perel; Maria Shugrina; Yoni Kasten; Or Litany; Gal Chechik; Sanja Fidler
- Web page: https://patentgrep.com/patent/US20250166288A1

## CPC

- G06T 15/08: Volume rendering
- G06T 15/20: Perspective computation
- G06T 2210/36: Level of detail
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Systems and methods of the present disclosure include providing higher levels of detail (LODs) for generated three-dimensional (3D) models, such as those represented by neural radiance fields (NeRFs). A 3D model may be presented to a user in which the user may request additional LODs, such as to zoom into the image or to receive information about features within the image. A request to generate finer levels of detail may include using one or more diffusion models to generate images at higher resolutions and/or to hallucinate finer details based on information extracted from the original image or text prompts. Newly generated images may then be added to a set of images associated with the 3D models to enable later model generation to have finer details.

## Claims

### Claim 1

A computer-implemented method, comprising:
determining a target level of detail for a three-dimensional (3D) volume;
generating, using an image generation network and based at least on a current view representing the 3D volume, an updated view representing the 3D volume at the target level of detail;
providing, responsive to the target level of detail, the updated view;
adding the updated view to a set of images associated with the 3D volume; and
updating the 3D volume, based at least on the updated view.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein the 3D volume is represented by a neural radiance field (NeRF).

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein the image generation network is a diffusion model conditioned on both text and images.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, further comprising:
receiving a prompt for the current view; and
providing, to a language model associated with the image generation network, the prompt.

### Claim 5 (depends on 4)

The computer-implemented method of claim 4, wherein the language model is a large language model (LLM) configured to generate a hierarchy of information based, at least, on the prompt.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein the image generation network is a super-resolution model conditioned on an image having a resolution less than a threshold.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, further comprising:
removing, upon receiving the updated view, one or more previous images from the set of images; and
updating a network associated with the 3D volume.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, wherein the target level of detail is associated with an input command from a user of an interactive environment.

### Claim 9 (depends on 1)

The computer-implemented method of claim 1, wherein the 3D volume is represented by a neural radiance field (NeRF), the method further comprising:
converting the NeRF to a mesh-based representation.

### Claim 10 (depends on 1)

The computer-implemented method of claim 1, further comprising:
receiving, at an associated LLM, a prompt requesting a hierarchy of information for an object associated with the 3D volume;
determining a plurality of sub-levels for the 3D volume based, at least, on the hierarchy of information; and
establishing an ordering for the plurality of sub-levels associated with a respective level for each sub-level of the plurality of sub-levels.

### Claim 11 (depends on 10)

The computer-implemented method of claim 10, further comprising:
storing the plurality of sub-levels;
providing, responsive to a first command, the object; and
providing, responsive to a second command, a sub-level of the plurality of sub-levels.

### Claim 12

A processor comprising:
one or more processing units to:
receive a request to generate an image using a neural radiance field (NeRF);
determine, from a prompt associated with the request, a target level of detail for the image;
determine that images generated using the NeRF will not meet the target level of detail;
generate, via one or more diffusion models, a new image at the target level of detail;
provide, responsive to the request, the new image; and
add the new image to a set of images associated with the NeRF.

### Claim 13 (depends on 12)

The processor of claim 12, wherein the prompt is a text prompt, and wherein the one or more processing units are further to:
provide the text prompt to a large language model (LLM);
receive, from the LLM, a command based, at least, on the text prompt; and
provide the command to the one or more diffusion models.

### Claim 14 (depends on 12)

The processor of claim 12, wherein the one or more diffusion models are conditioned on both text and images.

### Claim 15 (depends on 12)

The processor of claim 12, wherein at least one diffusion model of the one or more diffusion models include a super-resolution model conditioned on an image having a resolution less than a threshold.

### Claim 16 (depends on 12)

The processor of claim 12, wherein the processor is comprised in at least one of:
a system for performing simulation operations;
a system for performing simulation operations to test or validate autonomous machine applications;
a system for performing digital twin operations;
a system for performing light transport simulation;
a system for rendering graphical output;
a system for performing deep learning operations;
a system implemented using an edge device;
a system for generating or presenting virtual reality (VR) content;
a system for generating or presenting augmented reality (AR) content;
a system for generating or presenting mixed reality (MR) content;
a system incorporating one or more Virtual Machines (VMs);
a system for performing operations for a conversational AI application;
a system for performing operations for a generative AI application;
a system for performing operations using a language model;
a system for performing one or more generative content operations using a large language model (LLM);
a system implemented at least partially in a data center;
a system for performing hardware testing using simulation;
a system for performing one or more generative content operations using a language model;
a system for synthetic data generation;
a collaborative content creation platform for 3D assets; or
a system implemented at least partially using cloud computing resources.

### Claim 17

A system, comprising:
one or more processors comprising processing circuitry to generate an output image with a finer level of detail (LOD) than an input image generated using a neural radiance field (NeRF), and to update the NeRF using a set of images that includes the output image.

### Claim 18 (depends on 17)

The system of claim 17, wherein the output image is generated by one or more diffusion models responsive to a request.

### Claim 19 (depends on 17)

The system of claim 17, wherein the output image is at least one of a higher resolution image relative to the input image, or a hallucinated image.

### Claim 20 (depends on 17)

The system of claim 17, wherein the system comprises at least one of:
a system for performing simulation operations;
a system for performing simulation operations to test or validate autonomous machine applications;
a system for performing digital twin operations;
a system for performing light transport simulation;
a system for rendering graphical output;
a system for performing deep learning operations;
a system implemented using an edge device;
a system for generating or presenting virtual reality (VR) content;
a system for generating or presenting augmented reality (AR) content;
a system for generating or presenting mixed reality (MR) content;
a system incorporating one or more Virtual Machines (VMs);
a system for performing operations for a conversational AI application;
a system for performing operations for a generative AI application;
a system for performing operations using a language model;
a system for performing one or more generative content operations using a large language model (LLM);
a system implemented at least partially in a data center;
a system for performing hardware testing using simulation;
a system for performing one or more generative content operations using a language model;
a system for synthetic data generation;
a collaborative content creation platform for 3D assets; or
a system implemented at least partially using cloud computing resources.
