# US 11,908,115 B2 · Image transformation using interpretable transformation parameters

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,908,115 B2
- Document ID: US11908115B2
- Status: Granted
- Kind: B2
- Application number: 18/161,415
- Filed: 2023-01-30
- Published: 2023-06-01
- Granted: 2024-02-20
- Earliest claimed: 2023-01-30
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Diego Martin Arroyo; Alessio Tonioni; Federico Tombari
- Web page: https://patentgrep.com/patent/US11908115B2

## CPC

- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 11/00: Two-dimensional [2D] image generation
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0475: Generative networks
- G06N 3/08: Learning methods
- G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning
- G06N 3/094: Adversarial learning
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/20092: Interactive image processing based on input by user
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A computer-implemented method to perform image-to-image translation. The method can include obtaining one or more machine-learned generator models. The one or more machine-learned generator models can be configured to receive an input image and a user-specified conditioning vector that parameterizes one or more desired values for one or more defined characteristics of an output image. The one or more machine-learned generator models can be configured to perform, based at least in part on the user-specified conditioning vector, one or more transformations on the input image to generate the output image with the one or more desired values for the one or more defined characteristics. The method can include receiving the input image and the user-specified conditioning vector. The method can include generating, using the machine-learned generator model, an output image having the one or more desired values for the one or more characteristics.

## Claims

### Claim 1

A computer-implemented method to perform image-to-image translation, the method comprising:
obtaining, by a computing system comprising one or more computing devices, one or more machine-learned generator models configured to:
receive an input image and a user-specified conditioning vector that parameterizes one or more desired values for one or more defined characteristics of an output image; and
perform, based at least in part on the user-specified conditioning vector, one or more transformations on the input image to generate the output image with the one or more desired values for the one or more defined characteristics;
receiving, by the computing system, the input image and the user-specified conditioning vector; and
generating, by the computing system and using the one or more machine-learned generator models, an output image having the one or more desired values for the one or more defined characteristics.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein the one or more defined characteristics comprise one or more real-world characteristics.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein the user-specified conditioning vector describes a degree to which the one or more transformations are applied.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, wherein the one or more defined characteristics comprise:
light characteristics;
at least one light source location characteristics;
environmental characteristics;
color characteristics;
style characteristics; and
feature characteristics.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein the one or more machine-learned generator models comprise an encoder portion and decoder portion, wherein the user-specified conditioning vector is provided to the decoder portion of the one or more machine-learned generator models.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, further comprising:
training, by the computing system, at least one of one or more machine-learned discriminator models based at least in part on a difference between the output image and a ground truth associated with the output image.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein the input image is a digital photograph.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, wherein the output image comprises a different image style than the input image.

### Claim 9 (depends on 1)

The computer-implemented method of claim 1, wherein the output image comprises one or more modified features of the input image.