US 2026/0065429 A1Published application

Text to 3d content generation using timestep image re-sampling

AssigneeNVIDIA Corporation

InventorsJonathan Peter Lorraine, Jun Gao, Xiaohui Zeng, Cheng Xie, James Robert Lucas, Sanja Fidler

Life of this patent

Estimating the term…

Loading patent…

Abstract

Approaches presented herein provide systems and methods to reuse a rendered image for noising and denoising steps used for training one or more content generation systems. The reused rendered image may reduce computationally expensive processes, such as content generation and rendering, and enable multiple gradients to be compared using a common image that may be noised and then processed by one or more diffusion models to compute a gradient. The gradients may be combined and used to retrain the model, providing more training data with less variance between generating and rendering steps.

Claims20

Dependent claims

A computer-implemented method, comprising:

producing a first noised image by adding first noise at a first timestep to a reference image generated using a three-dimensional (3D) generative model;
computing a first gradient between the reference image and a first denoised image, the first denoised image being obtained by removing the first noise from the first noised image using a diffusion model and based on a received text prompt;
producing a second noised image by adding second noise at a second timestep to the reference image;
computing a second gradient between the reference image and a second denoised image, the second denoised image being obtained by removing the second noise from the second noised image using the diffusion model and based on the received text prompt;
and
generating a combined gradient based at least on the first gradient and the second gradient;
and
updating one or more parameters for the 3D generative model based at least on the combined gradient.

5 more elements

Dependencies are inferred from the claim text.