# US 10,481,044 B2 · Perception simulation for improved autonomous vehicle control

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,481,044 B2
- Document ID: US10481044B2
- Status: Granted
- Kind: B2
- Application number: 15/598,693
- Filed: 2017-05-18
- Published: 2018-11-22
- Granted: 2019-11-19
- Earliest claimed: 2017-05-18
- Assignees: Tusimple, Inc.
- Applicants: TuSimple
- Inventors: Xing Sun; Wutu Lin; Yufei Zhao; Liu Liu
- Web page: https://patentgrep.com/patent/US10481044B2

## CPC

- B60W 50/0205: {Diagnosing or detecting failures; Failure detection models}
- G01M 17/00: Testing of vehicles (testing fluid tightness G01M3/00; testing elastic properties of bodies or chassis, e.g. torsion-testing, G01M5/00; testing alignment of vehicle headlight devices G01M11/06; testing of engines G01M15/00)
- B60W 30/00: Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
- B60W 2050/0215: {Sensor drifts or sensor failures}
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G01M: TESTING STATIC OR DYNAMIC BALANCE OF MACHINES OR STRUCTURES; TESTING OF STRUCTURES OR APPARATUS, NOT OTHERWISE PROVIDED FOR

## Abstract

A system and method for real world autonomous vehicle perception simulation are disclosed. A particular embodiment includes: receiving perception data from a plurality of sensors of an autonomous vehicle; configuring the perception simulation operation based on a comparison of the perception data against ground truth data; generating simulated perception data by simulating errors related to the physical constraints of one or more of the plurality of sensors, and by simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors; and providing the simulated perception data to a motion planning system for the autonomous vehicle.

## Claims

### Claim 1

A system comprising:
a data processor; and
a perception simulation module, executable by the data processor, the perception simulation module being configured to perform a perception simulation operation for autonomous vehicles, the perception simulation operation being configured to:
receive perception data from a plurality of sensors of an autonomous vehicle;
configure the perception simulation operation based on a comparison of the perception data against ground truth data;
generate simulated perception data by simulating errors related to the physical constraints of one or more of the plurality of sensors, and by simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors, wherein simulating errors related to the physical constraints of one or more of the plurality of sensors includes applying an occlusion beyond which the one or more of the plurality of sensors cannot detect objects; and
provide the simulated perception data to a motion planning system for the autonomous vehicle.

### Claim 2 (depends on 1)

The system of claim 1 wherein simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors includes simulating errors in data produced by an image processing module.

### Claim 3

A method comprising:
receiving perception data from a plurality of sensors of an autonomous vehicle;
configuring the perception simulation operation based on a comparison of the perception data against ground truth data;
generating simulated perception data by simulating errors related to the physical constraints of one or more of the plurality of sensors, and by simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors, wherein simulating errors related to the physical constraints of one or more of the plurality of sensors includes applying an occlusion beyond which the one or more of the plurality of sensors cannot detect objects; and
providing the simulated perception data to a motion planning system for the autonomous vehicle.

### Claim 4 (depends on 3)

The method of claim 3 wherein simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors includes simulating errors in data produced by an image processing module.

### Claim 5

A non-transitory machine-useable storage medium embodying instructions which, when executed by a machine, cause the machine to:
receive perception data from a plurality of sensors of an autonomous vehicle;
configure the perception simulation operation based on a comparison of the perception data against ground truth data;
generate simulated perception data by simulating errors related to the physical constraints of one or more of the plurality of sensors, and by simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors, wherein simulating errors related to the physical constraints of one or more of the plurality of sensors includes applying an occlusion beyond which the one or more of the plurality of sensors cannot detect objects; and
provide the simulated perception data to a motion planning system for the autonomous vehicle.

### Claim 6 (depends on 5)

The non-transitory machine-useable storage medium of claim 5 wherein the ground truth data is produced by a ground truth data simulation system.

### Claim 7 (depends on 5)

The non-transitory machine-useable storage medium of claim 5 wherein simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors includes simulating errors in data produced by an image processing module.