Learning a 3d scene generation model from images of a self-similar scene

A system comprising:

• a memory component;
• and
• a processing device coupled to the memory component, the processing device to perform operations comprising:
• applying a scene generation model to a set of input 2D images of first content to generate a 3D remix scene, wherein the 3D remix scene includes second content that is different from the first content, wherein applying the scene generation model includes:
• generating a first 3D reference grid at a first resolution and a reference 2D image rendered from the 3D reference grid;
• configuring the scene generation model based on:
• performing a first comparison, using a 2D discriminator, of the 2D image and the 2D reference image;
• performing a second comparison, using a 3D discriminator, of a 3D feature grid at the first resolution generated by applying the scene generation model to the set of input 2D images to the 3D reference grid;
• and
• based at least on the first comparison and the second comparison, configuring one or more parameters of the scene generation model, wherein the 3D remix scene is generated using the configured scene generation model;
• and
• transmitting, for display via a user interface, the 3D remix scene.

A non-transitory computer-readable medium storing executable instructions, which when executed by a processing device, cause the processing device to perform operations comprising:

• applying a scene generation model to a set of input 2D images of first content to generate a 3D remix scene, wherein the 3D remix scene includes second content that is different from the first content, wherein applying the scene generation model includes:
• generating a first 3D reference grid at a first resolution;
• configuring the scene generation model based on:
• comparing, using at least a 2D discriminator and a 3D discriminator, the reference 3D grid and a 3D feature grid at the first resolution generated by applying the scene generation model to the set of input 2D images;
• and
• based at least on the comparing, configuring one or more parameters of the scene generation model, wherein the 3D remix scene is generated using the configured scene generation model;
• and
• transmitting, for display via a user interface, the 3D remix scene.