# US 2021/0080353 A1 · Perception simulation for improved autonomous vehicle control > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0080353 A1 - Document ID: US20210080353A1 - Status: Published application - Kind: A1 - Application number: 17/093,172 - Filed: 2020-11-09 - Published: 2021-03-18 - Earliest claimed: 2017-05-18 - Assignees: Tusimple, Inc. - Applicants: TuSimple, Inc. - Inventors: Xing SUN; Wutu LIN; Yufei ZHAO; Liu LIU - Web page: https://patentgrep.com/patent/US20210080353A1 ## CPC - B60W 50/0205: {Diagnosing or detecting failures; Failure detection models} - G01M 17/00: Testing of vehicles (testing fluid tightness G01M3/00; testing elastic properties of bodies or chassis, e.g. torsion-testing, G01M5/00; testing alignment of vehicle headlight devices G01M11/06; testing of engines G01M15/00) - B60W 30/00: Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units - B60W 2050/0215: {Sensor drifts or sensor failures} - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - G01M: TESTING STATIC OR DYNAMIC BALANCE OF MACHINES OR STRUCTURES; TESTING OF STRUCTURES OR APPARATUS, NOT OTHERWISE PROVIDED FOR ## Abstract A system and method for real world autonomous vehicle perception simulation are disclosed. A particular embodiment includes: configuring a sensor noise modeling module to produce simulated sensor errors or noise data with a configured degree, extent, and timing of simulated sensor errors or noise based on a set of modifiable parameters; using the simulated sensor errors or noise data to generate simulated perception data by simulating errors related to constraints of one or more of a plurality of sensors, and by simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors; and providing the simulated perception data to a motion planning system for the autonomous vehicle. ## Claims ### Claim 1 A system comprising: a data processor; and a perception simulation module, executable by the data processor, the perception simulation module being configured to: configure a sensor noise modeling module to produce simulated sensor errors or noise data with a configured degree, extent, and timing of simulated sensor errors or noise based on a set of modifiable parameters; use the simulated sensor errors or noise data to generate simulated perception data by simulating errors related to constraints of one or more of a plurality of sensors, and by simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors; and provide the simulated perception data to a motion planning system for an autonomous vehicle. ### Claim 2 (depends on 1) The system of claim 1 wherein the sensor noise modeling module being configured to produce simulated sensor errors or noise data includes simulating a false positive or false negative object detection. ### Claim 3 (depends on 1) The system of claim 1 wherein the sensor noise modeling module being configured to produce simulated sensor errors or noise data includes simulating noise in vehicle localization. ### Claim 4 (depends on 1) The system of claim 1 wherein the sensor noise modeling module being configured to produce simulated sensor errors or noise data includes simulating noise in lane detection. ### Claim 5 (depends on 1) The system of claim 1 wherein the sensor noise modeling module being configured to produce simulated sensor errors or noise data includes simulating errors in object type classification. ### Claim 6 (depends on 1) The system of claim 1 wherein the sensor noise modeling module being configured to produce simulated sensor errors or noise data includes simulating errors in object location. ### Claim 7 (depends on 1) The system of claim 1 wherein the sensor noise modeling module being configured to produce simulated sensor errors or noise data includes simulating errors in data produced by an image processing module. ### Claim 8 A method comprising: configuring a sensor noise modeling module to produce simulated sensor errors or noise data with a configured degree, extent, and timing of simulated sensor errors or noise based on a set of modifiable parameters; using the simulated sensor errors or noise data to generate simulated perception data by simulating errors related to constraints of one or more of a plurality of sensors, and by simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors; and providing the simulated perception data to a motion planning system for an autonomous vehicle. ### Claim 9 (depends on 8) The method of claim 8 wherein configuring the sensor noise modeling module to produce simulated sensor errors or noise data includes simulating a false positive or false negative result produced by a perception processing module. ### Claim 10 (depends on 8) The method of claim 8 wherein configuring the sensor noise modeling module to produce simulated sensor errors or noise data includes simulating noise produced by a perception sensor device. ### Claim 11 (depends on 8) The method of claim 8 wherein configuring the sensor noise modeling module to produce simulated sensor errors or noise data includes simulating noise in perception sensor devices and sensor data processing modules over a pre-determined time period. ### Claim 12 (depends on 8) The method of claim 8 including producing simulated perception data that corresponds to actual or simulated ground truth data. ### Claim 13 (depends on 8) The method of claim 8 including configuring or training the motion planning system for the autonomous vehicle or a vehicle simulation system. ### Claim 14 (depends on 8) The method of claim 8 wherein configuring the sensor noise modeling module to produce simulated sensor errors or noise data includes simulating errors corresponding to physical constraints of a perception sensor used with the autonomous vehicle. ### Claim 15 A non-transitory machine-useable storage medium embodying instructions which, when executed by a machine, cause the machine to: configure a sensor noise modeling module to produce simulated sensor errors or noise data with a configured degree, extent, and timing of simulated sensor errors or noise based on a set of modifiable parameters; use the simulated sensor errors or noise data to generate simulated perception data by simulating errors related to constraints of one or more of a plurality of sensors, and by simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors; and provide the simulated perception data to a motion planning system for an autonomous vehicle. ### Claim 16 (depends on 15) The non-transitory machine-useable storage medium of claim 15 wherein the plurality of sensors includes: image generating devices, cameras, distance measuring devices, light amplification by stimulated emission of radiation (laser) devices or light detection and ranging (LIDAR) devices, global positioning systems (GPS) devices, sound navigation and ranging (sonar) devices, or radio detection and ranging (radar) devices. ### Claim 17 (depends on 15) The non-transitory machine-useable storage medium of claim 15 wherein the sensor noise modeling module being configured to produce simulated sensor errors or noise data includes simulating a false positive or false negative object classification. ### Claim 18 (depends on 15) The non-transitory machine-useable storage medium of claim 15 wherein the sensor noise modeling module being configured to produce simulated sensor errors or noise data includes simulating noise in vehicle trajectory. ### Claim 19 (depends on 15) The non-transitory machine-useable storage medium of claim 15 wherein the sensor noise modeling module being configured to produce simulated sensor errors or noise data includes simulating noise in roadway detection. ### Claim 20 (depends on 15) The non-transitory machine-useable storage medium of claim 15 being configured to perform a calibration phase based on a comparison of real world perception data and ground truth data.