# US 2021/0342647 A1 · Semantic adversarial generation based function testing method in autonomous driving

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0342647 A1
- Document ID: US20210342647A1
- Status: Published application
- Kind: A1
- Application number: 16/862,630
- Filed: 2020-04-30
- Published: 2021-11-04
- Earliest claimed: 2020-04-30
- Assignees: Robert Bosch GmbH
- Applicants: Robert Bosch GmbH
- Inventors: Liang GOU; Lincan ZOU; Axel WENDT; Liu REN
- Web page: https://patentgrep.com/patent/US20210342647A1

## CPC

- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- G06F 18/2148: {characterised by the process organisation or structure, e.g. boosting cascade}
- G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques}
- G06N 3/045: Combinations of networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/047: Probabilistic or stochastic networks
- G06N 3/0475: Generative networks
- G06N 3/08: Learning methods
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06N 3/094: Adversarial learning
- G06T 9/002: {using neural networks}
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/82: using neural networks
- G06V 20/584: {of vehicle lights or traffic lights}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A system includes a camera configured to obtain image information from objects. The system also includes a processor in communication with the camera and programmed to receive an input data including the image information, encode the input via an encoder, obtain a latent variable defining an attribute of the input data, generate a sequential reconstruction of the input data utilizing at least the latent variable and an adversarial noise, obtain a residual between the input data and the sequential reconstruction utilizing a comparison of at least the input and the reconstruction to learn a mean shift in latent space, and output a mean shift indicating a test result of the input compared to the adversarial noise based on the comparison.

## Claims

### Claim 1

A system, comprising:
a camera configured to obtain image information from objects; and
a processor in communication with the camera and programmed to:
receive an input data including the image information;
encode the input via an encoder;
obtain a latent variable defining an attribute of the input data, generate a sequential reconstruction of the input data utilizing at least the latent variable and an adversarial noise, obtain a residual between the input data and the sequential reconstruction utilizing a comparison of at least the input and the reconstruction to learn a mean shift in latent space; and
output the mean shift indicating a test result of the input compared to the adversarial noise based on the comparison.

### Claim 2 (depends on 1)

The system of claim 1, wherein the output of the mean shift is sent to a generator and in response to the mean shift, the generator is configured to generate additional adversarial noises.

### Claim 3 (depends on 1)

The system of claim 1, wherein the system is further configured to output a mean shift vector associated with a gradient estimation of the comparison.

### Claim 4 (depends on 1)

The system of claim 1, wherein the adversarial noise includes a second input image.

### Claim 5 (depends on 1)

The system of claim 1, wherein the encoder is a neural network encoder.

### Claim 6 (depends on 1)

The system of claim 1, wherein the adversarial noise alters brightness, color, or contrast of the input data.

### Claim 7 (depends on 1)

The system of claim 1, wherein the input data includes video information obtained from the camera.

### Claim 8

An apparatus, comprising:
a processor programmed to:
receive an input data including at least an image of an object;
encode the input data via an encoder;
obtain a latent variable defining an attribute of the input data, generate a sequential reconstruction of the input data utilizing at least the latent variable and an adversarial noise, obtain a residual between the input data and the sequential reconstruction utilizing a comparison of at least the input and the reconstruction to learn a mean shift in latent space; and
output the mean shift indicating a test result of the input compared to the adversarial noise based on the comparison.

### Claim 9 (depends on 8)

The apparatus of claim 8, wherein the test result includes a numerical value.

### Claim 10 (depends on 8)

The apparatus of claim 8, wherein the output of the mean shift is sent to generate additional adversarial noise.

### Claim 11 (depends on 8)

The apparatus of claim 8, wherein the processor is further programmed to in a first sequence, compare the residual of a first and second latent variable, and in a second sequence, compare the residual between the first and second latent variable.

### Claim 12 (depends on 8)

The apparatus of claim 8, wherein the encoder is a neural network encoder.

### Claim 13 (depends on 8)

The apparatus of claim 8, wherein the adversarial noise includes a second input image that includes alterations to the input data including at least the image of the object.

### Claim 14

A computer-program product storing instructions which, when executed by a computer, cause the computer to:
receive an input data;
encode the input via an encoder;
obtain a latent variable defining an attribute of the input data, generate a sequential reconstruction of the input data utilizing at least the latent variable and an adversarial noise, obtain a residual between the input data and the sequential reconstruction utilizing a comparison of at least the input and the reconstruction of a mean shift in latent space; and
output a mean shift indicating a test result of the input compared to the adversarial noise based on the comparison.

### Claim 15 (depends on 14)

The computer-program product of claim 14, wherein the input includes an image received from a camera in communication with the computer.

### Claim 16 (depends on 14)

The computer-program product of claim 14, wherein the encoder includes a neural network.

### Claim 17 (depends on 14)

The computer-program product of claim 14, wherein the input data includes a data set of a plurality of images.

### Claim 18 (depends on 14)

The computer-program product of claim 14, wherein the computer-program product further includes instructions to update only one latent variable during each additional sequence.

### Claim 19 (depends on 14)

The computer-program product of claim 14, wherein the adversarial noise includes a second input image that includes alterations to the input data including at least the image of an object.

### Claim 20 (depends on 14)

The computer-program product of claim 14, wherein the adversarial noise alters brightness, color, or contrast of the input data.
