Compositional 3d-consistent freeview image generation with 3d blobs

A system, comprising:

• a non-transitory memory storage comprising instructions;
• and
• one or more processors in communication with the memory, wherein the one or more processors execute the instructions to:
• process an input that includes one or more three-dimensional (3D) blobs each representing an object in a scene and each having a corresponding text description of the object, using a diffusion model, to generate a two-dimensional (2D) image of the scene;
• and
• output the 2D image of the scene.

A method, comprising:

• at a device:
• generating a dataset of three-dimensional (3D) scene representations each comprised of one or more 3D blobs that each represent an object in a scene and that each have a corresponding text description of the object;
• and
• training a diffusion model, using the dataset, to generate two-dimensional (2D) images of scenes from input 3D scene representations comprised of object-level 3D blobs and corresponding object-level text descriptions.