# US 2025/0166136 A1 · Editing Control Of Material Properties With Diffusion Models

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0166136 A1
- Document ID: US20250166136A1
- Status: Published application
- Kind: A1
- Application number: 18/957,367
- Filed: 2024-11-22
- Published: 2025-05-22
- Earliest claimed: 2023-11-22
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Mark Jeffrey Matthews; Prafull Sharma; Dmitry Lagun; Xuhui Jia; Yuanzhen Li; Varun Jampani; William Tafel Freeman
- Web page: https://patentgrep.com/patent/US20250166136A1

## CPC

- G06T 11/00: Two-dimensional [2D] image generation
- G06T 5/60: using machine learning, e.g. neural networks
- G06T 5/70: Denoising; Smoothing
- G06T 5/73: Deblurring; Sharpening
- G06T 11/60: Creating or editing images; Combining images with text
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Provided are systems and methods for controlling material attributes such as roughness, metallic, albedo, and transparency in real images. This method leverages the generative prior of text-to-image models known for their photorealistic capabilities, offering an alternative to traditional rendering pipelines. As one example, the technology can be used to alter the appearance of an object in an image, making it appear more metallic or changing its roughness to create a more matte or glossy finish. This can be particularly useful in various fields where the ability to manipulate the appearance of products in images can be a powerful tool.

## Claims

### Claim 1

A computer-implemented method to train a diffusion model to perform visual editing of material properties, the method comprising:
obtaining, by a computing system comprising one or more computing devices, a pair of training images, wherein the pair of training images comprises a target image and a context image, wherein the target image and the context image each depict a shared object, and wherein a material property of the shared object differs between the target image and the context image;
adding, by the computing system, a set of noise to the target image to obtain a noised target image;
processing, by the computing system, the noised target image with a denoising diffusion model that is conditioned on the context image to generate a denoising prediction;
evaluating, by the computing system, a loss function that evaluates the denoising prediction relative to the set of noise; and
modifying, by the computing system, one or more values of one or more parameters of the denoising diffusion model based on the loss function.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein the denoising diffusion model is further conditioned on a textual prompt that indicates an edit to the material property.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein the denoising diffusion model is further conditioned on a scalar edit value for the material property, wherein the scalar edit value corresponds to the material property exhibited by the target image.

### Claim 4 (depends on 3)

The computer-implemented method of claim 3, wherein the scalar edit value is provided to the denoising diffusion model in an additional input channel.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein the pair of training images were generated using a physically-based renderer, wherein the physically-based renderer comprises a shader, and wherein the shader was supplied with different values for the material property of the shared object when rendering the target image and the context image.

### Claim 6 (depends on 5)

The computer-implemented method of claim 5, wherein the shader was supplied with a value of zero for the material property of the shared object when rendering the context image.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein the material property comprises one or more of roughness, metallic property, albedo, and transparency.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, wherein the denoising diffusion model comprises a latent diffusion model and the set of noise is added in a latent space.

### Claim 9

A computing system comprising:
one or more processors; and
one or more non-transitory computer-readable media that collectively store:
a denoising diffusion model configured to perform visual edits to a material property of an object depicted in a context image;
instructions that, when executed by the computing system, cause the computing system to perform operations, the operations comprising:
receiving the context image;
processing an input with the denoising diffusion model to generate an edited image, wherein the denoising diffusion model is conditioned on the context image, and wherein the edited image depicts the object with a modified material property; and
providing the edited image as an output.

### Claim 10 (depends on 1, 9)

The computing system of claim 9, wherein the denoising diffusion model has been trained according to the method of claim 1.

### Claim 11 (depends on 9)

The computing system of claim 9, wherein the input comprises a noise input and one or both of:
a textual prompt that describes an edit to the material property; and
a scalar edit value for the material property.

### Claim 12 (depends on 11)

The computing system of claim 11, wherein the input comprises the scalar edit value, and wherein the scalar edit value is provided to the denoising diffusion model in an additional input channel.

### Claim 13 (depends on 9)

The computing system of any of claim 9, wherein the material property comprises one or more of roughness, metallic property, albedo, and transparency.

### Claim 14

One or more non-transitory computer-readable media that collectively store a denoising diffusion model configured to perform visual editing of material properties, the diffusion model having been trained by performance of training operations, the training operations comprising:
obtaining, by a computing system comprising one or more computing devices, a pair of training images, wherein the pair of training images comprises a target image and a context image, wherein the target image and the context image each depict a shared object, and wherein a material property of the shared object differs between the target image and the context image;
adding, by the computing system, a set of noise to the target image to obtain a noised target image;
processing, by the computing system, the noised target image with the denoising diffusion model that is conditioned on the context image to generate a denoising prediction;
evaluating, by the computing system, a loss function that evaluates the denoising prediction relative to the set of noise; and
modifying, by the computing system, one or more values of one or more parameters of the denoising diffusion model based on the loss function.

### Claim 15 (depends on 14)

The one or more non-transitory computer-readable media of claim 14, wherein the denoising diffusion model is further conditioned on a textual prompt that indicates an edit to the material property.

### Claim 16 (depends on 14)

The one or more non-transitory computer-readable media of claim 14, wherein the denoising diffusion model is further conditioned on a scalar edit value for the material property, wherein the scalar edit value corresponds to the material property exhibited by the target image.

### Claim 17 (depends on 16)

The one or more non-transitory computer-readable media of claim 16, wherein the scalar edit value is provided to the denoising diffusion model in an additional input channel.

### Claim 18 (depends on 14)

The one or more non-transitory computer-readable media of claim 14, wherein the pair of training images were generated using a physically-based renderer, wherein the physically-based renderer comprises a shader, and wherein the shader was supplied with different values for the material property of the shared object when rendering the target image and the context image.

### Claim 19 (depends on 14)

The one or more non-transitory computer-readable media of claim 14, wherein the material property comprises one or more of roughness, metallic property, albedo, and transparency.

### Claim 20 (depends on 14)

The one or more non-transitory computer-readable media of claim 14, wherein the denoising diffusion model comprises a latent diffusion model and the set of noise is added in a latent space.
