Utilizing diffusion models with three-dimensional gaussian point clouds to generate multi-view images from a digital image

A system comprising:

• a memory component comprising a diffusion model;
• and
• a processing device coupled to the memory component, wherein the processing device is configured to perform operations comprising:
• receiving a prompt image depicting a visual entity from a first viewpoint and an input camera condition;
• and
• generating three-dimensional (3D) Gaussian point clouds representing the visual entity from a second viewpoint by:
• generating a noisy prompt image from the prompt image;
• and
• generating the 3D Gaussian point clouds representing the visual entity from the second viewpoint from denoising the noisy prompt image utilizing the diffusion model conditioned on input camera condition.

A computer-implemented method comprising:

• identifying a training image depicting a visual entity from a first viewpoint and a noisy training image depicting the visual entity from a second viewpoint;
• and
• training a diffusion model to generate three-dimensional (3D) Gaussian point clouds representing the visual entity from novel viewpoints by:
• denoising the noisy training image utilizing the diffusion model to generate predicted 3D Gaussian point clouds depicting the visual entity from the second viewpoint;
• determining a measure of loss between a rasterized image of the predicted 3D Gaussian point clouds depicting the visual entity from the second viewpoint and a ground truth image corresponding to the noisy training image;
• and
• modifying parameters of the diffusion model utilizing the measure of loss.