Text-Driven 3D Gaussian Splat Shape and Color Stylization

A non-transitory computer-readable medium comprising instructions, the instructions, when executed by a computer system, causing the computer system to perform operations including:

• receiving an image depicting a scene captured by a camera of a client device;
• determining a pose of the camera when the image was captured;
• applying a stylized three-dimensional (3D) model to generate a stylized view of the scene from the pose, wherein the stylized 3D model was generated by:
• obtaining an unstylized 3D model of the scene;
• rendering, using the unstylized 3D model of the scene, a set of unstylized images of the scene having poses corresponding to a representative trajectory;
• generating, based on the representative trajectory, a set of consistently styled images from the set of unstylized images using a diffusion model;
• and
• generating the stylized 3D model using the set of consistently styled images;
• and
• causing display of the stylized view by the client device.

A computer-implemented method comprising:

• receiving an image depicting a scene captured by a camera of a client device;
• determining a pose of the camera when the image was captured;
• applying a stylized three-dimensional (3D) model to generate a stylized view of the scene from the pose, wherein the stylized 3D model is configured by parameters including a first strength parameter for controlling an amount by which color in the image is stylized and a second strength parameter for controlling an amount by which depth in the image is stylized;
• and
• causing display of the stylized view by the client device.