# US 12,214,801 B2 · Generating autonomous vehicle testing data through perturbations and adversarial loss functions

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,214,801 B2
- Document ID: US12214801B2
- Status: Granted
- Kind: B2
- Application number: 17/528,549
- Filed: 2021-11-17
- Published: 2022-05-19
- Granted: 2025-02-04
- Earliest claimed: 2020-11-17
- Assignees: Aurora Operations, Inc.
- Applicants: Aurora Operations, Inc.
- Inventors: Jingkang Wang; Ava Alison Pun; Xuanyuan Tu; Mengye Ren; Abbas Sadat; Sergio Casas; Sivabalan Manivasagam; Raquel Urtasun
- Web page: https://patentgrep.com/patent/US12214801B2

## CPC

- B60W 60/0011: {involving control alternatives for a single driving scenario, e.g. planning several paths to avoid obstacles}
- G06N 3/006: based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
- B60W 60/0013: {specially adapted for occupant comfort}
- B60W 60/00276: {for two or more other traffic participants}
- G06N 3/08: Learning methods
- G06N 3/086: using evolutionary algorithms, e.g. genetic algorithms or genetic programming
- G06N 3/09: Supervised learning
- G06N 3/092: Reinforcement learning
- G06N 3/094: Adversarial learning
- G06N 3/126: Evolutionary algorithms, e.g. genetic algorithms or genetic programming
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N 20/00: Machine learning
- B60W 2420/408: Radar; Laser, e.g. lidar
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Techniques for generating testing data for an autonomous vehicle (AV) are described herein. A system can obtain sensor data descriptive of a traffic scenario. The traffic scenario can include a subject vehicle and actors in an environment. Additionally, the system can generate a perturbed trajectory for a first actor in the environment based on perturbation values. Moreover, the system can generate simulated sensor data. The simulated sensor data can include data descriptive of the perturbed trajectory for the first actor in the environment. Furthermore, the system can provide the simulated sensor data as input to an AV control system. The AV control system can be configured to process the simulated sensor data to generate an updated trajectory for the subject vehicle in the environment. Subsequently, the system can evaluate an adversarial loss function based on the updated trajectory for the subject vehicle to generate an adversarial loss value.

## Claims

### Claim 1

A computer-implemented method for generating testing data for an autonomous vehicle, the method comprising:
(a) obtaining sensor data descriptive of a traffic scenario comprising a subject vehicle and one or more actors in an environment of the subject vehicle;
(b) generating a perturbed trajectory for a first actor of the one or more actors in the environment based on one or more perturbation values that are selected from a defined perturbation search space by: creating a set of physically feasible trajectories for the first actor; generating an initial perturbed trajectory for the first actor based on the one or more perturbation values; and projecting the initial perturbed trajectory onto the set of physically feasible trajectories to generate the perturbed trajectory;
(c) generating simulated sensor data comprising data descriptive of the perturbed trajectory for the first actor in the environment;
(d) providing the simulated sensor data as input to an autonomous vehicle control system configured to process the simulated sensor data to generate an updated trajectory for the subject vehicle in the environment; and
(e) optimizing an adversarial loss function based on the updated trajectory for the subject vehicle to generate an adversarial loss value.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein the adversarial loss function comprises one or more of the following:
an imitation-learning cost term that encourages the updated trajectory to deviate from an original trajectory of the subject vehicle in the traffic scenario;
a cumulative collision cost term that encourages the perturbation values to cause the subject vehicle to collide with the one or more actors; and
a comfort cost term that encourages the updated trajectory to have lane violations, high acceleration, or jerk.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein (b) comprises:
generating a defined perturbation search space;
selecting the one or more perturbation values from the defined perturbation search space by using a black-box optimization technique; and
wherein the perturbed trajectory for the first actor is generated based on the selection of the one or more perturbation values.

### Claim 4 (depends on 3)

The computer-implemented method of claim 3, wherein (b) comprises employing a reinforcement learning agent to select the perturbation values from the perturbation search space.

### Claim 5 (depends on 3)

The computer-implemented method of claim 3, wherein the perturbation search space comprises initial state values and a set of change in curvatures and acceleration values over a number of timesteps.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein (b) comprises selecting the one or more perturbation values from a defined perturbation search space based at least in part on a set of historical observations associated with previously selected perturbation values, and wherein the method further comprises:
(f) adding the one or more perturbation values and the adversarial loss value to the set of historical observations; and
after (f), repeating (a)-(d).

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein the perturbed trajectory is parameterized as a series of kinematic bicycle model states.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, wherein the autonomous vehicle control system is a simulated AV control system.

### Claim 9 (depends on 1)

The computer-implemented method of claim 1, wherein (b) comprises selecting a closest reachable actor as the first actor.

### Claim 10 (depends on 1)

The computer-implemented method of claim 1, wherein (b) comprises ensuring that the perturbed trajectory avoids collision with: (i) one or more existing trajectories of one or more other actors in the environment; or (ii) an initial trajectory for the subject vehicle in the environment.

### Claim 11 (depends on 1)

The computer-implemented method of claim 1, wherein:
the sensor data comprises light detection and ranging (LiDAR) data; and
the simulated sensor data comprises simulated LiDAR data.

### Claim 12 (depends on 11)

The computer-implemented method of claim 11, wherein (d) comprises:
removing LiDAR points within a bounding box associated with the first actor,
simulating new background LiDAR points after removing the LiDAR points within the bounding box;
inserting simulated LiDAR points based on the perturbed trajectory for the first actor; and
removing LiDAR points included within a synthesized LiDAR shadow for one or more occluded regions based on the perturbed trajectory for the first actor.

### Claim 13 (depends on 1)

The computer-implemented method of claim 1, wherein the sensor data comprises real-world sensor data previously collected by one or more physical sensors in the environment.

### Claim 14 (depends on 1)

The computer-implemented method of claim 1, wherein the autonomous vehicle control system comprises one or more machine-learned models, and wherein the method further comprises updating one or more values of one or more parameters of the one or more machine-learned models based on the adversarial loss function.

### Claim 15 (depends on 1)

The computer-implemented method of claim 1, wherein (b) comprises performing a black-box optimization technique to select the one or more perturbation values from a defined perturbation search space, and wherein the black-box optimization technique comprises one of the following:
Bayesian optimization;
a genetic algorithm;
random search; or
a gradient estimation method.

### Claim 16 (depends on 1)

The computer-implemented method of claim 1, wherein the autonomous vehicle control system comprises one of the following:
an end-to-end imitation learning system;
a neural motion planner;
a jointly learnable behavior and trajectory planning system; or
a perceive, predict, and plan system.

### Claim 17

A computing system comprising:
one or more processors; and
one or more computer-readable medium storing instructions that when executed by the one or more processors cause the computing system to perform operations, the operations comprising:
(a) obtaining sensor data descriptive of a traffic scenario comprising a subject vehicle and one or more actors in an environment;
(b) selecting one or more perturbation values from a defined perturbation search space;
(c) generating a perturbed trajectory for a first actor of the one or more actors in the environment based on the one or more perturbation values by: creating a set of physically feasible trajectories for the first actor; generating an initial perturbed trajectory for the first actor based on the one or more perturbation values; and projecting the initial perturbed trajectory onto the set of physically feasible trajectories to generate the perturbed trajectory; and
(d) generating simulated sensor data descriptive of the perturbed trajectory for the first actor in the environment.

### Claim 18 (depends on 17)

The computing system of claim 17, wherein (b) comprises selecting the one or more perturbation values from the defined perturbation search space based at least in part on a set of historical observations associated with previously selected perturbation values, and wherein the operations further comprise:
(e) providing the simulated sensor data as input to an autonomous vehicle control system configured to process the simulated sensor data to generate an updated trajectory for the subject vehicle in the environment;
(f) evaluating an adversarial loss function based on the updated trajectory for the subject vehicle to generate an adversarial loss value;
(g) adding the one or more perturbation values and the adversarial loss value to the set of historical observations; and
after (g), repeating (a)-(d).

### Claim 19

An autonomous vehicle control system comprising:
one or more processors; and
one or more non-transitory computer-readable medium storing: (i) one or more machine-learned models, wherein the one or more machine-learned models have been learned via performance of machine learning algorithms on one or more training examples comprising simulated sensor data; and (ii) instructions for execution by the one or more processors to cause the one or more processors to perform operations to generate the simulated sensor data, the operations comprising:
(a) obtaining sensor data descriptive of a traffic scenario comprising a subject vehicle and one or more actors in an environment;
(b) selecting one or more perturbation values from a defined perturbation search space;
(c) generating a perturbed trajectory for a first actor of the one or more actors in the environment based on the one or more perturbation values by: creating a set of physically feasible trajectories for the first actor; generating an initial perturbed trajectory for the first actor based on the one or more perturbation values; and projecting the initial perturbed trajectory onto the set of physically feasible trajectories to generate the perturbed trajectory; and
(d) generating simulated sensor data descriptive of the perturbed trajectory for the first actor in the environment.