# US 12,112,503 B2 · Method and apparatus for calibrating external parameters of image acquisition device, device and storage medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,112,503 B2 - Document ID: US12112503B2 - Status: Granted - Kind: B2 - Application number: 17/356,649 - Filed: 2021-06-24 - Published: 2021-10-14 - Granted: 2024-10-08 - Earliest claimed: 2020-07-20 - Assignees: Beijing Baidu Netcom Science Technology Co., Ltd. - Applicants: Beijing Baidu Netcom Science and Technology Co., LTD - Inventors: Caimao Yuan; Pengbin Yang; Fengze Han; Fei Ni; Yang Yang - Web page: https://patentgrep.com/patent/US12112503B2 ## CPC - G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration - G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)} - H04N 17/002: {for television cameras} - H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums - B60W 50/06: Improving the dynamic response of the control system, e.g. improving the speed of regulation or avoiding hunting or overshoot - B60W 60/001: {Planning or execution of driving tasks} - B60W 2420/403: Image sensing, e.g. optical camera - G06T 2207/30208: Marker matrix - G06T 2207/30244: Camera pose - G06T 2207/30252: Vehicle exterior; Vicinity of vehicle - G06T 2207/30256: Lane; Road marking - Y02T 10/40: Engine management systems - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - Y02T: CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION ## Abstract The present application discloses a method and an apparatus for calibrating external parameters of an image acquisition device, a device and a storage medium, and relates to the technical field of vehicle positioning. The specific implementation solution is as follows: acquiring an image captured by the image acquisition device; determining a plurality of marking points on a calibration plate in the image; determining a first conversion matrix based on coordinates of each of the marking points in the image pixel coordinate system and the calibration plate coordinate system; determining a second conversion matrix based on coordinates of at least one wheel grounding point of the vehicle in the calibration plate coordinate system and the vehicle body coordinate system; and determining external parameters of the image acquisition device by using the first conversion matrix and the second conversion matrix. ## Claims ### Claim 1 A method for calibrating external parameters of an image acquisition device which is provided on a body side of a vehicle, and a field of view of which is downward, and a calibration plate being arranged on the ground at the body side of the vehicle and being within the field of view of the image acquisition device, the method comprising: acquiring an image captured by the image acquisition device; determining a plurality of marking points on a calibration plate in the image; determining a first conversion matrix from an image pixel coordinate system to a calibration plate coordinate system based on coordinates of each of the marking points in the image pixel coordinate system and the calibration plate coordinate system; determining a second conversion matrix from the calibration plate coordinate system to a vehicle body coordinate system based on coordinates of at least one wheel grounding point of the vehicle in the calibration plate coordinate system and the vehicle body coordinate system; and determining external parameters of the image acquisition device by using the first conversion matrix and the second conversion matrix; wherein, determining the coordinates of the wheel grounding point in the calibration plate coordinate system, comprises: determining the coordinates of two wheel grounding points on one side of the vehicle in the calibration plate coordinate system; and determining the coordinates of two wheel grounding points on the other side of the vehicle in the calibration plate coordinate system by using the determined coordinates and a vehicle model of the vehicle. ### Claim 2 (depends on 1) The method according to claim 1, wherein, the calibration plate comprises a ChArUco calibration plate. ### Claim 3 (depends on 2) The method according to claim 2, wherein, the determining the plurality of marking points on the calibration plate in the image, comprises: determining a plurality of chessboard corner points on the ChArUco calibration plate. ### Claim 4 (depends on 1) A method for evaluating accuracy errors of external parameters of an image acquisition device, the external parameters of the image acquisition device being determined by the method of claim 1, the method comprising: evaluating accuracy errors of the first conversion matrix and/or the second conversion matrix; wherein, wherein, evaluating an accuracy error of the first conversion matrix comprises: calculating first coordinates of a calibration point in the calibration plate coordinate system by using the coordinates of the calibration point in a calibration plate in the image pixel coordinate system and the first conversion matrix; and calculating a distance between the first coordinates and real coordinates of the calibration point in the calibration plate coordinate system, to evaluate the accuracy error of the first conversion matrix; and wherein, evaluating an accuracy error of the second conversion matrix comprises: calculating second coordinates of a wheel grounding point of the vehicle in the vehicle body coordinate system by using the coordinates of the wheel grounding point of the vehicle in the calibration plate coordinate system and the second conversion matrix; and calculating a distance between the second coordinates and real coordinates of the wheel grounding point in the vehicle body coordinate system, to evaluate the accuracy error of the second conversion matrix. ### Claim 5 (depends on 4) The method according to claim 4, wherein the calibration point in the calibration plate comprises a calibration point in the calibration plate which is not used in determining the first conversion matrix; and the wheel grounding point of the vehicle comprises a wheel grounding point among the wheel grounding points of the vehicle that is not used in determining the second conversion matrix. ### Claim 6 (depends on 4) The method according to claim 4, wherein the calibration plate comprises a ChArUco calibration plate. ### Claim 7 (depends on 4) The method according to claim 4, further comprising: using two image acquisition devices arranged on the same side of the vehicle to respectively fit two corresponding straight lines; and determining an angle between the two corresponding straight lines to evaluate the accuracy errors of the external parameters of the image acquisition devices; wherein, using the image acquisition devices to fit the straight lines comprises: determining a plurality of calibration points at the same straight line on the calibration plate, in the image captured by the image acquisition device; determining, for each calibration point, the coordinates of the calibration point in the vehicle body coordinate system, by using the coordinates of the calibration point in the calibration plate coordinate system, the first conversion matrix and the second conversion matrix; and fitting the plurality of calibration points into a straight line by using the coordinates of the plurality of calibration points in the vehicle body coordinate system. ### Claim 8 (depends on 4) The method according to claim 4, further comprising: determining a high-accuracy region of the image acquisition device by using the accuracy error of the first conversion matrix. ### Claim 9 An apparatus for calibrating external parameters of an image acquisition device which is provided on a body side of a vehicle, and a field of view of which is downward, and a calibration plate being arranged on the ground at the body side of the vehicle and being within the field of view of the image acquisition device, the apparatus comprising: a processor and a memory for storing one or more computer programs executable by the processor, wherein when executing at least one of the computer programs, the processor is configured to perform operations comprising: acquiring an image captured by the image acquisition device; determining a plurality of marking points on a calibration plate in the image; determining a first conversion matrix from an image pixel coordinate system to a calibration plate coordinate system based on coordinates of each of the marking points in the image pixel coordinate system and the calibration plate coordinate system; determining a second conversion matrix from the calibration plate coordinate system to a vehicle body coordinate system based on coordinates of at least one wheel grounding point of the vehicle in the calibration plate coordinate system and the vehicle body coordinate system; and determining external parameters of the image acquisition device by using the first conversion matrix and the second conversion matrix; wherein, when executing at least one of the computer programs, the processor is configured to further perform operations comprising: determining the coordinates of two wheel grounding points on one side of the vehicle in the calibration plate coordinate system; and determining the coordinates of two wheel grounding points on the other side of the vehicle in the calibration plate coordinate system by using the determined coordinates and a vehicle model of the vehicle. ### Claim 10 (depends on 9) The apparatus according to claim 9, wherein, the calibration plate comprises a ChArUco calibration plate. ### Claim 11 (depends on 10) The apparatus according to claim 10, wherein, when executing at least one of the computer programs, the processor is configured to further perform operations comprising determining a plurality of chessboard corner points on the ChArUco calibration plate. ### Claim 12 (depends on 4, 9) The apparatus according to claim 9, wherein, when executing at least one of the computer programs, the processor is configured to perform the method of claim 4. ### Claim 13 (depends on 12) The apparatus according to claim 12, wherein, when executing at least one of the computer programs, the processor is configured to further perform operations comprising: utilizing a calibration point in the calibration plate which is not used in determining the first conversion matrix; and utilizing a wheel grounding point among the wheel grounding points of the vehicle which is not used in determining the second conversion matrix. ### Claim 14 (depends on 12) The apparatus according to claim 12, wherein the calibration plate comprises a ChArUco calibration plate. ### Claim 15 (depends on 12) The apparatus according to claim 12, wherein, when executing at least one of the computer programs, the processor is configured to further perform operations comprising: using two image acquisition devices arranged on the same side of the vehicle to respectively fit two corresponding straight lines; and determining an angle between the two corresponding straight lines to evaluate the accuracy errors of the external parameters of the image acquisition devices, wherein, when executing at least one of the computer programs, the processor is configured to further perform operations comprising: determining a plurality of calibration points at the same straight line on the calibration plate, in the image captured by the image acquisition device; determining, for each calibration point, the coordinates of the calibration point in the vehicle body coordinate system, by using the coordinates of the calibration point in the calibration plate coordinate system, the first conversion matrix and the second conversion matrix; and fitting the plurality of calibration points into a straight line by using the coordinates of the plurality of calibration points in the vehicle body coordinate system. ### Claim 16 (depends on 12) The apparatus according to claim 12, wherein, when executing at least one of the computer programs, the processor is configured to further perform operations comprising: determining a high-accuracy region of the image acquisition device, by using the accuracy error of the first conversion matrix. ### Claim 17 (depends on 1) A non-transitory computer readable storage medium storing computer instructions, wherein, the computer instructions cause a computer to execute the method according to claim 1. ### Claim 18 (depends on 4, 17) The non-transitory computer readable storage medium according to claim 17, wherein, the computer instructions cause a computer to execute the method according to claim 4.