# US 2019/0317510 A1 · System and method for full-stack verification of autonomous agents

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0317510 A1
- Document ID: US20190317510A1
- Status: Published application
- Kind: A1
- Application number: 15/955,320
- Filed: 2018-04-17
- Published: 2019-10-17
- Earliest claimed: 2018-04-17
- Assignees: Toyota Research Institute, Inc.
- Applicants: Toyota Research Institute, Inc.
- Inventors: German ROS SANCHEZ
- Web page: https://patentgrep.com/patent/US20190317510A1

## CPC

- B60W 60/00276: {for two or more other traffic participants}
- G06N 3/084: Backpropagation, e.g. using gradient descent
- B60W 60/001: {Planning or execution of driving tasks}
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06N 3/008: based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
- G06N 5/02: Knowledge representation; Symbolic representation
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A method for full-stack verification of autonomous agents includes training a neural network to learn a noise model associated with an object detection module of an autonomous agent system of an autonomous vehicle. The method also includes replacing the object detection module of the autonomous agent system with the neural network and a sensory input of the object detection module with ground truth information to apply a surrogate function to the ground truth information. The method further includes verifying the autonomous agent system including the trained neural network to apply the surrogate function in response to the ground truth information to simulate sensor information data to at least a planner module of the autonomous agent system. The method also includes controlling a behavior of the autonomous vehicle using the verified autonomous agent system including the object detection module.

## Claims

### Claim 1

A method for full-stack verification of autonomous agents, comprising:
training a neural network to learn a noise model associated with an object detection module of an autonomous agent system of an autonomous vehicle;
replacing the object detection module of the autonomous agent system with the neural network and a sensory input of the object detection module with ground truth information to apply a surrogate function to the ground truth information;
verifying the autonomous agent system including the trained neural network to apply the surrogate function in response to the ground truth information to simulate sensor information data to at least a planner module of the autonomous agent system; and
controlling a behavior of the autonomous vehicle using the verified autonomous agent system including the object detection module.

### Claim 2 (depends on 1)

The method of claim 1, in which training comprises:
training the neural network to learn the surrogate function to mimic the behavior of a sensor module Mi by learning the noise model associated with the sensor module Mi; and
repeating the training for other sensor modules of the autonomous agent system to generate a set of the surrogate function for full-stack verification of the autonomous agent system.

### Claim 3 (depends on 1)

The method of claim 1, in which training further comprises:
running the object detection module M for a set of training data De to determine a noisy output ŷi; and
estimating the noise model of the object detection module M based on the noisy output ŷi and a ground truth yi of the object detection module M.

### Claim 4 (depends on 3)

The method of claim 3, further comprising:
defining an ontology for the object detection module M; and
creating a finite set of values for each attribute of a vector of measurements.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
improving the planner module of the autonomous vehicle prior to controlling the behavior of the autonomous vehicle according to simulated operation of the autonomous vehicle during corner cases or atypical situations presented by a simulated sensor information data.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
improving a controller module of the autonomous vehicle prior to controlling the behavior of the autonomous vehicle according to simulated operation of the autonomous vehicle during corner cases or atypical situations presented by a simulated sensor information data.

### Claim 7 (depends on 6)

The method of claim 6, in which verifying comprises:
certifying collision avoidance of the autonomous vehicle during extreme situations presented by the simulated sensor information data.

### Claim 8 (depends on 7)

The method of claim 7, further comprising:
operating the autonomous vehicle using a certified collision avoidance module in a pipeline of the autonomous agent system to improve the operation of the autonomous vehicle during the real-world extreme situations and/or atypical real-world situations.

### Claim 9

A non-transitory computer-readable medium having program code recorded thereon for full-stack verification of autonomous agents, the program code being executed by a processor and comprising:
program code to train a neural network to learn a noise model associated with an object detection module of an autonomous agent system;
program code to replacing the object detection module of the autonomous agent system with the neural network and a sensory input of the object detection module with ground truth information to apply a surrogate function to the ground truth information;
program code to verifying the autonomous agent system including the trained neural network to apply the surrogate function in response to the ground truth information to simulate sensor information data to at least a planner module of the autonomous agent system; and
program code to controlling a behavior of an autonomous vehicle using the verified autonomous agent system including the object detection module.

### Claim 10 (depends on 9)

The non-transitory computer-readable medium of claim 9, in which program code to train comprises:
program code to train the neural network to learn the surrogate function to mimic the behavior of a sensor module Mi by learning the noise model associated with the sensor module Mi; and
program code to repeat the program code to training the neural network for other sensor modules of the autonomous agent system to generate a set of the surrogate function for the full-stack verification of the autonomous agent system.

### Claim 11 (depends on 9)

The non-transitory computer-readable medium of claim 9, in which program code to train further comprises:
program code to run the object detection module M for a set of training data De to determine a noisy output ŷi; and
program code to estimate the noise model of the object detection module M based on the noisy output ŷi, and a ground truth yi of the object detection module M.

### Claim 12 (depends on 11)

The non-transitory computer-readable medium of claim 11, further comprising:
program code to define an ontology for the object detection module M; and
program code to create a finite set of values for each attribute of a vector of measurements.

### Claim 13 (depends on 9)

The non-transitory computer-readable medium of claim 9, further comprising:
program code to optimize the planner module of the autonomous vehicle according to simulated operation of the autonomous vehicle during corner cases or atypical situations presented by a simulated sensor information data.

### Claim 14 (depends on 9)

The non-transitory computer-readable medium of claim 9, further comprising:
program code to optimize a controller module of the autonomous vehicle according to simulated operation of the autonomous vehicle during corner cases or atypical situations presented by a simulated sensor information data.

### Claim 15 (depends on 9)

The non-transitory computer-readable medium of claim 9, further comprising:
program code to certify collision avoidance of the autonomous vehicle during extreme situations presented by a simulated sensor information data.

### Claim 16

A system for full-stack verification of an autonomous vehicle system, comprising:
a neural network configured to learn a noise model associated with an object detection module of the autonomous vehicle system, in which the object detection module of the autonomous vehicle system is replaced with the neural network and a sensory input of the autonomous vehicle system is replaced with ground truth information during the full-stack verification, the neural network further configured to apply a surrogate function to the ground truth information to simulate sensor information data to at least a planner module of the autonomous vehicle system for the full-stack verification during corner cases and/or atypical situations presented by a simulated sensor information data; and
a controller configured to improve behavior of an autonomous vehicle during real-world extreme situations.

### Claim 17 (depends on 16)

The system for the full-stack verification of the autonomous vehicle system of claim 16, in which the planner module is further configured to improve route planning for the autonomous vehicle based on predicted actions of surrounding agents during the real-world extreme situations.

### Claim 18 (depends on 16)

The system for the full-stack verification of the autonomous vehicle system of claim 16, in which the controller is further configured to improve operation of the autonomous vehicle during the real-world extreme situations.

### Claim 19 (depends on 16)

The system for the full-stack verification of the autonomous vehicle system of claim 16, in which the controller further comprises a collision avoidance module, in which operation of the collision avoidance module is certified as correct and safe using the extreme situations presented by the simulated sensor information data.

### Claim 20 (depends on 16)

The system for the full-stack verification of the autonomous vehicle system of claim 16, in which the ground truth information comprises two-dimensional (2D) and/or three-dimensional (3D) bounding boxes of detected objects in an area surrounding the autonomous vehicle.
