# US 2021/0241497 A1 · Generation of controlled attribute-based images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0241497 A1
- Document ID: US20210241497A1
- Status: Published application
- Kind: A1
- Application number: 16/778,906
- Filed: 2020-01-31
- Published: 2021-08-05
- Earliest claimed: 2020-01-31
- Assignees: Adobe Inc.
- Applicants: Adobe Inc.
- Inventors: Shradha Agrawal; Deepak Pai; Dhanya Raghu
- Web page: https://patentgrep.com/patent/US20210241497A1

## CPC

- G06T 11/00: Two-dimensional [2D] image generation
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F 18/22: Matching criteria, e.g. proximity measures
- G06F 18/24: Classification techniques
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/047: Probabilistic or stochastic networks
- G06N 3/0475: Generative networks
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06N 3/09: Supervised learning
- G06N 3/094: Adversarial learning
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/82: using neural networks
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T 2210/16: Cloth
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Embodiments of the present disclosure are directed towards generating images conditioned on a desired attribute. In particular, an attribute-based image generation system can use a directional-GAN architecture to generate images conditioned on a desired attribute. A latent vector and a desired attribute are received. A feature subspace is determined for the latent vector using a latent-attribute linear classifier trained to determine a relationship between the latent vector and the desired attribute. An image is generated using the latent vector such that the image contains the desired attribute. In embodiments, where the feature space differs from a desired feature subspace, a directional vector is applied to the latent vector that shifts the latent vector from the feature subspace to the desired feature subspace. This modified latent vector is then used during generation of the image.

## Claims

### Claim 1

A computer-implemented method, comprising:
receiving a latent vector and a first desired attribute;
determining a feature subspace for the latent vector using one or more latent-attribute linear classifiers trained to determine a first relationship between the latent vector and the first desired attribute;
receiving a second desired attribute;
determining a second relationship between the latent vector and the second desired attribute; and
generating an image, using the latent vector, wherein the image contains the desired first and second desired attributes.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, further comprising:
comparing the feature subspace with a desired feature subspace, the desired feature subspace associated with the desired attribute; and
based on the comparison indicating that the feature subspace differs from the desired feature subspace, applying a directional vector to the latent vector that shifts the latent vector from the feature subspace to the desired feature subspace.

### Claim 3 (depends on 2)

The computer-implemented method of claim 2, wherein the directional vector is a delta plus a signed distance from the latent vector to a hyperplane separating the desired feature subspace from other feature subspaces, the delta moving the latent vector over the hyperplane.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, wherein the one or more latent-attribute linear classifiers are progressively trained at more than one resolution.

### Claim 5 (depends on 4)

The computer-implemented method of claim 4, further comprising:
comparing the second feature space with a second desired feature space, the second desired feature space associated with the second desired attribute; and
based on the comparison indicating that the second feature space differs from the second desired feature space, applying a second directional weight to the latent vector that shifts the latent vector from the second feature space to the second desired feature space.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein the latent vector is a randomly generated latent vector and the desired attribute is input by a user.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein training the latent-attribute linear classifier comprises:
receiving a set of classified images and corresponding latent vectors; and
learning a linear separating plane between the feature subspace and another feature subspace, wherein the feature subspace is associated with a first attribute value and the other feature space is associated with a second attribute value.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, wherein a set of classified images are generated using a trained image generation neural network that are classified using an image-attribute classifier.

### Claim 9 (depends on 1)

The computer-implemented method of claim 1, wherein a trained image generation neural network that generated full-body images of people is used for generating the image that contains the desired attributes.

### Claim 10

One or more computer storage media storing computer-useable instructions that, when used by one or more computing devices, cause the one or more computing devices to perform operations comprising:
receiving a latent vector and a desired attribute;
determining a feature subspace for the latent vector using a latent-attribute linear classifier progressively trained at more than one resolution to determine a relationship between the latent vector and the desired attribute; and
generating an image, using the latent vector, wherein the image contains the desired attribute.

### Claim 11 (depends on 10)

The one or more computer storage media of claim 10, the operations further comprising:
comparing the feature subspace with a desired feature subspace, the desired feature subspace associated with the desired attribute; and
based on the comparison indicating that the feature subspace differs from the desired feature subspace, applying a directional vector to the latent vector that shifts the latent vector from the feature subspace to the desired feature subspace.

### Claim 12 (depends on 11)

The one or more computer storage media of claim 11, wherein the magnitude of the directional vector is a delta plus a signed distance from the latent vector to a hyperplane separating the feature subspace from the desired feature subspace, the delta moving the latent vector over the hyperplane.

### Claim 13 (depends on 10)

The one or more computer storage media of claim 10, further comprising:
receiving a second desired attribute; and
determining a second feature space for the latent vector using a second latent-attribute linear classifier trained to determine a second relationship between the latent vector and the second desired attribute.

### Claim 14 (depends on 13)

The one or more computer storage media of claim 13, further comprising:
comparing the second feature space with a second desired feature space, the second desired feature space associated with the second desired attribute; and
based on the comparison indicating that the second feature space differs from the second desired feature space, applying a second directional weight to the latent vector that shifts the latent vector from the second feature space to the second desired feature space.

### Claim 15 (depends on 10)

The one or more computer storage media of claim 10, wherein the latent vector is a randomly generated latent vector and the desired attribute is input by a user.

### Claim 16 (depends on 10)

The one or more computer storage media of claim 10, wherein training the trained latent-attribute linear classifier comprises:
receiving a set of classified images and corresponding latent vectors; and
learning a linear separating plane between the feature space and another feature space, wherein the feature space is associated with a first attribute and the other feature space is associated with a second attribute.

### Claim 17 (depends on 10)

The one or more computer storage media of claim 10, wherein a set of classified images are generated using a trained image generation neural network that are classified using an image-attribute classifier.

### Claim 18 (depends on 10)

The one or more computer storage media of claim 10, wherein a trained image generation neural network that generated full-body images of people is used for generating the image that contains the desired attribute.

### Claim 19

A computing system comprising:
means for receiving a latent vector;
means for classifying the latent vector to be in a feature subspace as a desired attribute;
means for applying a directional vector to the latent vector that shifts the latent vector from one feature subspace to another feature subspace associated with the desired attribute; and
means for generating an image, using the latent vector, wherein the image contains the desired attribute.

### Claim 20 (depends on 19)

The computing system of claim 19, wherein the desired attribute is a pose.