3d scene generation using sparse voxel latent diffusion models

Abstract

Approaches presented herein provide for generation of high-resolution 3D representations of objects or scenes at scale. The sparse nature of the 3D data can be leveraged using a sparse voxel hierarchical representation where only values for “active” voxels are stored. A generative model, such as a diffusion model, can be run through a number of iterations to generate latent representations with different levels of detail, which can be decoded into sparse voxel hierarchical representations of the object at these different levels of detail. In a first iteration, this can correspond to a coarse representation of a object that can be used to condition the generation of a refined representation of that object. A generated high-resolution sparse voxel grid representation of an object can be used with a mesh and texture to render a realistic view of a 3D scene including that object.

Dependencies are inferred from the claim text.