# US 2022/0277187 A1 · Concept-based adversarial generation method with steerable and diverse semantics

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0277187 A1
- Document ID: US20220277187A1
- Status: Published application
- Kind: A1
- Application number: 17/188,863
- Filed: 2021-03-01
- Published: 2022-09-01
- Earliest claimed: 2021-03-01
- Assignees: Robert Bosch GmbH
- Applicants: Robert Bosch GmbH
- Inventors: Zijie Wang; Liang Gou; Wenbin He; Liu Ren
- Web page: https://patentgrep.com/patent/US20220277187A1

## CPC

- G06N 3/045: Combinations of networks
- G06V 40/172: {Classification, e.g. identification}
- G06F 18/24: Classification techniques
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/047: Probabilistic or stochastic networks
- G06N 3/0475: Generative networks
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06N 3/094: Adversarial learning
- G06V 10/82: using neural networks
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Methods and systems for performing concept-based adversarial generation with steerable and diverse semantics. One system includes an electronic processor configured to access an input image. The electronic processor is also configured to perform concept-based semantic image generation based on the input image. The electronic processor is also configured to perform concept-based semantic adversarial learning using a set of semantic latent spaces generated as part of performing the concept-based semantic image generation. The electronic processor is also configured to generate an adversarial image based on the concept-based semantic adversarial learning. The electronic processor is also configured to test a target model using the adversarial image.

## Claims

### Claim 1

A system for performing concept-based adversarial generation with steerable and diverse semantics, the system comprising:
an electronic processor configured to
access an input image,
perform concept-based semantic image generation based on the input image,
perform concept-based semantic adversarial learning using a set of semantic latent spaces generated as part of performing the concept-based semantic image generation,
generate an adversarial image based on the concept-based semantic adversarial learning, and
test a target model using the adversarial image.

### Claim 2 (depends on 1)

The system of claim 1, wherein the electronic processor is configured to perform concept-based semantic image generation using disentangled representation learning and generative adversarial learning.

### Claim 3 (depends on 1)

The system of claim 1, wherein the electronic processor is configured to perform concept-based semantic image generation by
training an image generator with disentangled continuous and discrete representation, and
training a generative adversarial network based on a semantic latent space disentangled by the image generator.

### Claim 4 (depends on 3)

The system of claim 3, wherein the image generator is a variational autoencoder.

### Claim 5 (depends on 3)

The system of claim 3, wherein the training the image generator includes
training an encoder configured to
learn a disentangled latent space with continuous and discrete encodings, and
align the disentangled latent space with semantic features as the semantic latent space.

### Claim 6 (depends on 3)

The system of claim 3, wherein training the image generator also includes
training a decoder configured to generate a new image from the semantic latent space.

### Claim 7 (depends on 3)

The system of claim 3, wherein the electronic processor is configured to train the generative adversarial network with random input and a vector sampled from the semantic latent space.

### Claim 8 (depends on 1)

The system of claim 1, wherein the concept-based semantic adversarial learning is performed as a black-box adversarial learning approach.

### Claim 9 (depends on 1)

The system of claim 1, wherein the electronic processor is configured to perform the concept-based semantic adversarial learning by
analyzing the set of semantic latent spaces, and
identifying at least one semantic latent space,
wherein the adversarial image is generated based on the at least one semantic latent space.

### Claim 10 (depends on 9)

The system of claim 9, wherein the adversarial image has at least one semantic feature different than the input image.

### Claim 11 (depends on 1)

The system of claim 1, wherein the adversarial image will fail the target model.

### Claim 12 (depends on 11)

The system of claim 11, wherein the target model is an attribute classifier.

### Claim 13 (depends on 1)

The system of claim 1, wherein the electronic processor is further configured to generate a performance summary of the target model for display.

### Claim 14

A method for performing concept-based adversarial generation with steerable and diverse semantics, the method comprising:
accessing an input image;
performing, with an electronic processor, concept-based semantic image generation based on the input image;
performing, with the electronic processor, concept-based semantic adversarial learning using a set of semantic latent spaces generated as part of performing the concept-based semantic image generation;
generating, with the electronic processor, an adversarial image based on the concept-based semantic adversarial learning;
testing, with the electronic processor, a target model using the adversarial image; and
generating, with the electronic processor, a performance summary of the target model for display.

### Claim 15 (depends on 14)

The method of claim 14, wherein performing concept-based semantic image generation includes performing the concept-based semantic image generation using disentangled representation learning and generative adversarial learning.

### Claim 16 (depends on 14)

The method of claim 14, wherein performing the concept-based semantic image generation includes
training an image generator with disentangled continuous and discrete representation, and
training a generative adversarial network based on a semantic latent space disentangled by the image generator.

### Claim 17 (depends on 16)

The method of claim 16, wherein the training the image generator includes
training an encoder configured to
learn a disentangled latent space with continuous and discrete encodings, and
align the disentangled latent space with semantic features as the semantic latent space; and
training a decoder configured to generate a new image from the semantic latent space.

### Claim 18 (depends on 14)

The method of claim 14, wherein performing the concept-based semantic adversarial learning includes
analyzing the set of semantic latent spaces, and
identifying at least one semantic latent space,
wherein the adversarial image is generated based on the at least one semantic latent space.

### Claim 19

A non-transitory, computer-readable medium storing instructions that, when executed by an electronic processor, perform a set of functions, the set of functions comprising:
accessing an input image;
performing concept-based semantic image generation based on the input image;
performing concept-based semantic adversarial learning using a set of semantic latent spaces generated as part of performing the concept-based semantic image generation;
generating an adversarial image based on the concept-based semantic adversarial learning;
testing a target model using the adversarial image; and
generating a performance summary of the target model for display.

### Claim 20 (depends on 19)

The computer-readable medium of claim 19, wherein performing the concept-based semantic image generation includes
training an image generator with disentangled continuous and discrete representation, and
training a generative adversarial network based on a semantic latent space disentangled by the image generator.
