# US 11,443,524 B2 · Systems and methods for validating sensor calibration > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,443,524 B2 - Document ID: US11443524B2 - Status: Granted - Kind: B2 - Application number: 16/695,291 - Filed: 2019-11-26 - Published: 2020-06-11 - Granted: 2022-09-13 - Earliest claimed: 2018-12-10 - Assignees: Motional AD LLC - Applicants: Motional AD LLC - Inventors: Mauro Aguiar; Noam Weinstein; Eric Wolff; Matthias Sapuan; Hsun-Hsien Chang; Philipp Robbel; Maurilio Di Cicco; Guchan Ozbilgin; Bishwamoy Sinha Roy; Yifan Yang; Akshay Bhagat; David Butterworth; Andrew J. Eick; Alok Sharma; Junqing Wei - Web page: https://patentgrep.com/patent/US11443524B2 ## CPC - G01S 7/4802: {using analysis of echo signal for target characterisation; Target signature; Target cross-section} - G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads - G01S 7/4972: {Alignment of sensor} - G01S 17/86: Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders - G01S 17/89: for mapping or imaging - G02B 6/0055: {Reflecting element, sheet or layer} - G05D 1/0088: {characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00)} - G05D 1/81: Handing over between on-board automatic and on-board manual control - G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Among other things, we describe systems and method for validating sensor calibration. For validating calibration of a system of sensors having several types of sensors, an object may be configured to have a substantially reflective portion such that the sensors can isolate the substantially reflective portion, and their sensor data can be compared to determine, if the detected locations of the substantially reflective portion by each sensor are aligned. For calibrating a system of sensors, an object having known calibration features can be used and detected by each sensor, and the detected data can be compared to known calibration data associated with the object to determine if each sensor is correctly calibrated. ## Claims ### Claim 1 A system, comprising: at least one object comprising at least one calibration feature; and a vehicle comprising: a plurality of sensors configured to detect the at least one calibration feature of the at least one object while the at least one object and the vehicle move relative to each other, each sensor of the plurality of sensors having a plurality of parameters, wherein the plurality of parameters comprises a plurality of intrinsic parameters and a plurality of extrinsic parameters; a computer-readable medium storing computer-executable instructions; and at least one processor configured to be communicatively coupled to the plurality of sensors and execute the instructions stored on the computer-readable medium; wherein when the at least one processor executes the instructions, the at least one processor carries out operations to: receive known calibration feature data associated with the at least one calibration feature, wherein the known calibration feature data is known to be accurate; receive, from each sensor of the plurality of sensors, detected feature data associated with the at least one calibration feature; compare the received known feature data with the received detected feature data; generate, for each sensor of the plurality of sensors, a calibration error value based on the comparison of the received known feature data with the received detected feature data; and determine that at least one sensor of the plurality of sensors is incorrectly calibrated when the calibration error value corresponding with the at least one sensor is greater than a first calibration error threshold. ### Claim 2 (depends on 1) The system of claim 1, wherein when the at least one processor executes the instructions, the at least one processor further carries out operations to compute a corrected parameter for at least one intrinsic parameter associated with the at least one sensor determined to be incorrectly calibrated, wherein computing the corrected parameter is at least partially based on the determining that the at least one sensor is incorrectly calibrated. ### Claim 3 (depends on 2) The system of claim 2, wherein when the at least one processor executes the instructions, the at least one processor further carries out operations to modify the at least one intrinsic parameter based on the corrected parameter. ### Claim 4 (depends on 2) The system of claim 2, wherein each of the plurality of sensors comprise at least one monitoring device configured to detect intrinsic data corresponding with the at least one intrinsic parameter, and wherein when the at least one processor executes the instructions, the at least one processor further carries out operations to: receive intrinsic data corresponding with the at least one intrinsic parameter, and compute a corrected parameter for the at least one intrinsic parameter associated with the at least one sensor determined to be incorrectly calibrated, wherein the computing of the corrected parameter is for the at least one intrinsic parameter is at least partially based on the received intrinsic data. ### Claim 5 (depends on 1) The system of claim 1, wherein the plurality of intrinsic parameters comprises at least one of: an operating frequency, a field of view, a beam width, a beam power, and a signal-to-noise ratio. ### Claim 6 (depends on 1) The system of claim 1, wherein when the at least one processor executes the instructions, the at least one processor further carries out operations to determine that the at least one sensor determined to be calibrated incorrectly is in a failure state when the calibration error value corresponding with the at least one sensor is greater than a second calibration error threshold. ### Claim 7 (depends on 6) The system of claim 6, wherein when the at least one processor executes the instructions, the at least one processor carries out operations to compute a corrected parameter for at least one extrinsic parameter associated with the at least one sensor determined to be in the failure state, wherein the computing of the corrected parameter for the at least one extrinsic parameter is at least partially based on the determining that the at least one sensor is in the failure state. ### Claim 8 (depends on 1) The system of claim 1, wherein the plurality of extrinsic parameters include at least one of: the locations of the sensors as it is mounted on the vehicle, the distance of the sensor with respect to the other sensors, the angle of the sensor, the noise level caused by the environment, and the ambient brightness of the environment. ### Claim 9 (depends on 1) The system of claim 1, wherein the at least one object comprises three surfaces defining three mutually perpendicular planes, each surface of the three surfaces comprising at least one calibration feature. ### Claim 10 (depends on 1) The system of claim 1, wherein the at least one object comprises a fixed road feature. ### Claim 11 (depends on 1) The system of claim 1, wherein the plurality of sensors comprises a first sensor having a first type and a second sensor having a second type. ### Claim 12 (depends on 1) The system of claim 1, wherein the plurality of sensors comprises at least one RADAR sensor, and the least one object comprises an inner metal core being detectable by the at least one RADAR sensor. ### Claim 13 (depends on 1) The system of claim 1, wherein the plurality of sensors comprises at least one light detection and ranging sensor and at least one RADAR sensor, and the at least one object comprises an augmented portion, the augmented portion being substantially detectable by the at least one light detection and ranging sensor and substantially undetectable by the at least one RADAR sensor. ### Claim 14 (depends on 1) The system of claim 1, wherein the known feature data comprises at least one of: information received from a remote sensor, information received from another vehicle, information received from the one or more sensors at a prior time, information determined by a pre-calibrated sensor of the one or more sensor. ### Claim 15 (depends on 1) The system of claim 1, wherein the known feature data comprises information received from a remote sensor mounted on a second vehicle. ### Claim 16 (depends on 1) The system of claim 1, wherein the known feature data comprises information received from a sensor of a first type and the plurality of sensors comprises at least one sensor of a second type being different from the first type. ### Claim 17 (depends on 1) The system of claim 1, wherein when the at least one processor executes the instructions, the at least one processor carries out operations to cause the vehicle to cease operation if the calibration error value corresponding with the at least one sensor determined to be incorrectly calibrated is greater than a calibration error threshold. ### Claim 18 (depends on 1) The system of claim 1, wherein when the at least one processor executes the instructions, the at least one processor carries out operations to notify a remote technician if the calibration error value corresponding with the at least one sensor determined to be incorrectly calibrated is greater than a calibration error threshold. ### Claim 19 (depends on 1) The system of claim 1, wherein when the at least one processor executes the instructions, the at least one processor carries out operations to navigate the vehicle to a service location if the calibration error value corresponding with the at least one sensor determined to be incorrectly calibrated is greater than a calibration error threshold. ### Claim 20 (depends on 1) The system of claim 1, wherein when the at least one processor executes the instructions, the at least one processor carries out operations to disable the at least one sensor determined to be incorrectly calibrated if the calibration error value corresponding with the at least one sensor is greater than a calibration error threshold. ### Claim 21 A method, comprising: moving a vehicle and at least one object relative to each other, the vehicle comprising a plurality of sensors and the at least one object comprising at least one calibration feature; detecting, with the plurality of sensors, the at least one calibration feature of the at least one object while the at least one object and the vehicle are moving relative to each other, wherein each sensor of the plurality of sensors comprises a plurality of parameters, the plurality of parameters comprising a plurality of intrinsic parameters and a plurality of extrinsic parameters; receiving known calibration feature data associated with the at least one calibration feature, wherein the known calibration feature data is known to be accurate; receiving, from each sensor of the plurality of sensors, detected feature data associated with the at least one calibration feature; comparing the received known feature data with the received detected feature data; generating, for each sensor of the plurality of sensors, a calibration error value based on the comparison of the received known feature data with the received detected feature data; and determining that at least one sensor of the plurality of sensors is incorrectly calibrated when the calibration error value corresponding with the at least one sensor is greater than a first calibration error threshold. ### Claim 22 A non-transitory computer program product storing instructions that, when executed by at least one programmable processor, cause the at least one programmable computer processor to perform operations comprising: moving a vehicle and at least one object relative to each other, the vehicle comprising a plurality of sensors and the at least one object comprising at least one calibration feature; detecting, with the plurality of sensors, the at least one calibration feature of the at least one object while the at least one object and the vehicle are moving relative to each other, wherein each sensor of the plurality of sensors comprises a plurality of parameters, the plurality of parameters comprising a plurality of intrinsic parameters and a plurality of extrinsic parameters; receiving known calibration feature data associated with the at least one calibration feature, wherein the known calibration feature data is known to be accurate; receiving, from each sensor of the plurality of sensors, detected feature data associated with the at least one calibration feature; comparing the received known feature data with the received detected feature data; generating, for each sensor of the plurality of sensors, a calibration error value based on the comparison of the received known feature data with the received detected feature data; and determining that at least one sensor of the plurality of sensors is incorrectly calibrated when the calibration error value corresponding with the at least one sensor is greater than a first calibration error threshold.