US 12,608,873 B2 · Interactive neural field editing in content generation systems and applications

Abstract

Systems and methods of the present disclosure include interactive editing for generated three-dimensional (3D) models, such as those represented by neural radiance fields (NeRFs). A 3D model may be presented to a user in which the user may identify one or more localized regions for editing and/or modification. The localized regions may be selected and a corresponding 3D volume for that region may be provided to one or more generative networks, along with a prompt, to generate new content for the localized regions. Each of the original NeRF and the newly generated NeRF for the new content may then be combined into a single NeRF for a combined 3D representation with the original content and the localized modifications.

Claims17

Dependent claims

A computer-implemented method, comprising:

receiving an input corresponding to content for a selected portion of a 3D volume of a first neural radiance field (NeRF) corresponding to a scene;
receiving an interaction to the first NeRF, the interaction corresponding to the selected portion;
determining, based at least on a parameter of the interaction, a depth for the selected portion;
generating the selected portion using the depth;
generating a second NeRF of the selected portion based, at least, on the input;
and
generating an output NeRF by blending the first NeRF and the second NeRF.

4 more elements

A processor comprising:

one or more processing units to:
generate, via one or more diffusion models, a scene representation as a neural radiance field (NeRF);
receive an input corresponding to a portion of the NeRF;
generate, based on an indication, a modified NeRF for the portion, the modified NeRF including one or more content portions different from the NeRF;
combine the modified NeRF and the NeRF to form an updated NeRF, wherein the portion of the NeRF is a three-dimensional (3D) representation generated from a two-dimensional (2D) input and one or more properties of the 2D input;
and
provide a visual representation of the updated NeRF.

4 more elements

A system, comprising:

one or more processors comprising processing circuitry to generate an output representation of a three-dimensional (3D) scene as a combined neural radiance field (NeRF), the combined NeRF including an original volume and a modified volume added to the original volume, the modified volume being generated for a selected volume of the original volume based, at least, on an input corresponding to content for the modified volume, wherein the modified volume is computed from a second input corresponding to a two-dimensional area and a property of the second input.

Dependencies are inferred from the claim text.

Facts

Application

18/479,261Filed 2 Oct 2023

Published

US 2025/0111588 A13 Apr 2025

Granted

21 Apr 2026

Term ends

Not estimatedThe source records don’t support an estimate.

Classification5 codes

G06T 15/06
Three-dimensional [3D] image rendering: ray-tracing
G06T 15/005
Three-dimensional [3D] image rendering: general purpose rendering architectures
G06T 2207/20084
Special algorithmic details: artificial neural networks [ANN]
G06T 2210/56
Indexing scheme for image generation or computer graphics: particle system, point based geometry or rendering

Interactive neural field editing in content generation systems and applications

Life of this patent

Interactive neural field editing in content generation systems and applications

Life of this patent

Abstract

Claims17