Editing Control Of Material Properties With Diffusion Models

A computing system comprising:

• one or more processors;
• and
• one or more non-transitory computer-readable media that collectively store:
• a denoising diffusion model configured to perform visual edits to a material property of an object depicted in a context image;
• instructions that, when executed by the computing system, cause the computing system to perform operations, the operations comprising:
• receiving the context image;
• processing an input with the denoising diffusion model to generate an edited image, wherein the denoising diffusion model is conditioned on the context image, and wherein the edited image depicts the object with a modified material property;
• and
• providing the edited image as an output.

One or more non-transitory computer-readable media that collectively store a denoising diffusion model configured to perform visual editing of material properties, the diffusion model having been trained by performance of training operations, the training operations comprising:

• obtaining, by a computing system comprising one or more computing devices, a pair of training images, wherein the pair of training images comprises a target image and a context image, wherein the target image and the context image each depict a shared object, and wherein a material property of the shared object differs between the target image and the context image;
• adding, by the computing system, a set of noise to the target image to obtain a noised target image;
• processing, by the computing system, the noised target image with the denoising diffusion model that is conditioned on the context image to generate a denoising prediction;
• evaluating, by the computing system, a loss function that evaluates the denoising prediction relative to the set of noise;
• and
• modifying, by the computing system, one or more values of one or more parameters of the denoising diffusion model based on the loss function.