# US 10,769,793 B2 · Method for pitch angle calibration based on 2D bounding box and its 3D distance for autonomous driving vehicles (ADVs) > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,769,793 B2 - Document ID: US10769793B2 - Status: Granted - Kind: B2 - Application number: 15/955,644 - Filed: 2018-04-17 - Published: 2019-10-17 - Granted: 2020-09-08 - Earliest claimed: 2018-04-17 - Assignees: Baidu USA LLC - Applicants: Baidu USA LLC - Inventors: I-Kuei Chen - Web page: https://patentgrep.com/patent/US10769793B2 ## CPC - G06T 7/12: Edge-based segmentation - B60W 40/11: Pitch movement - B60W 60/001: {Planning or execution of driving tasks} - G05D 1/0219: {ensuring the processing of the whole working surface} - G05D 1/024: {in combination with a laser (lasers per se H01S)} - G06T 7/73: using feature-based methods - B60W 2420/403: Image sensing, e.g. optical camera - G06T 2207/20081: Training; Learning - G06T 2207/30241: Trajectory - G06T 2207/30244: Camera pose - G06T 2207/30252: Vehicle exterior; Vicinity of vehicle - G06T 2210/12: Bounding box - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES ## Abstract In one embodiment, a system of an ADV perceives an object in a perspective view of the ADV using a plurality of sensors mounted on the ADV including capturing an image of the object. The system determines a two-dimensional (2D) bounding box for the object and a perceived distance from the ADV to the object. The system searches, within an image space of the image, a dynamic pitch angle based on the 2D bounding box and the perceived distance from the ADV to the object by modifying at least a previous extrinsic calibration value to calculate a distance. The system determines a distance from the ADV to a point on a ground plane of the perspective view of the ADV based on the determined dynamic pitch angle. The system generates a driving trajectory based on the determined distance for the point on the ground plane to control the ADV. ## Claims ### Claim 1 A computer-implemented method for operating an autonomous driving vehicle (ADV), the method comprising: perceiving an object in a perspective view of the ADV using a plurality of sensors mounted on the ADV including capturing an image of the object using a monocular image capturing device; determining a two-dimensional (2D) bounding box for the object and a perceived distance from the ADV to the object based on the image of the object; searching, within an image space of the image of the object, for a dynamic pitch angle based on the 2D bounding box and the perceived distance from the ADV to the object by modifying at least a previous extrinsic calibration value to calculate a distance value; determining a distance from the ADV to a point on a ground plane of the perspective view of the ADV based on the determined dynamic pitch angle; and generating a driving trajectory based on the determined distance for the point on the ground plane to control the ADV. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein modifying at least a previous extrinsic calibration value includes modifying a pitch angle associated with a previous extrinsic calibration value for the monocular image capturing device. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, wherein the 2D bounding box for the object is determined by: applying a machine learning algorithm model to the image capturing the object in the perspective view; and determining a dimension of the 2D bounding box bounding the object based on the machine learning algorithm model. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, further comprises determining a ground point of contact of the 2D bounding box to calculate a distance from the ADV to the object based on the ground point of contact for the 2D bounding box. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, wherein searching for the dynamic pitch angle comprises calculating the distance from the ADV to the object based on formula D=h/tan(alpha+beta), where beta is the dynamic pitch angle to be calculated, D is the calculated distance, alpha is a previous calibration angle from a camera optical axis to the ground, and h is an estimated height of the object. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, wherein the dynamic pitch angle is searched using a binary search algorithm until the calculated distance from the ADV to the object is within a second predetermined threshold of the perceived distance from the ADV to the object. ### Claim 7 (depends on 1) The computer-implemented method of claim 1, wherein the dynamic pitch angle is searched using a greedy search algorithm until the calculated distance from the ADV to the object is within a second predetermined threshold of the perceived distance from the ADV to the object. ### Claim 8 A non-transitory machine-readable medium having instructions stored therein, which when executed by one or more processors, cause the one or more processors to perform operations, the operations comprising: perceiving an object in a perspective view of the ADV using a plurality of sensors mounted on the ADV including capturing an image of the object using a monocular image capturing device; determining a two-dimensional (2D) bounding box for the object and a perceived distance from the ADV to the object based on the image of the object; searching, within an image space of the image of the object, for a dynamic pitch angle based on the 2D bounding box and the perceived distance from the ADV to the object by modifying at least a previous extrinsic calibration value to calculate a distance value; determining a distance from the ADV to a point on a ground plane of the perspective view of the ADV based on the determined dynamic pitch angle; and generating a driving trajectory based on the determined distance for the point on the ground plane to control the ADV. ### Claim 9 (depends on 8) The non-transitory machine-readable medium of claim 8, wherein modifying at least a previous extrinsic calibration value includes modifying a pitch angle associated with a previous extrinsic calibration value for the monocular image capturing device. ### Claim 10 (depends on 8) The non-transitory machine-readable medium of claim 8, wherein the 2D bounding box for the object is determined by: applying a machine learning algorithm model to the image capturing the object in the perspective view; and determining a dimension of the 2D bounding box bounding the object based on the machine learning algorithm model. ### Claim 11 (depends on 8) The non-transitory machine-readable medium of claim 8, further comprises determining a ground point of contact of the 2D bounding box to calculate a distance from the ADV to the object based on the ground point of contact for the 2D bounding box. ### Claim 12 (depends on 8) The non-transitory machine-readable medium of claim 8, wherein searching for the dynamic pitch angle comprises calculating the distance from the ADV to the object based on formula D=h/tan(alpha+beta), where beta is the dynamic pitch angle to be calculated, D is the calculated distance, alpha is a previous calibration angle from a camera optical axis to the ground, and h is an estimated height of the object. ### Claim 13 (depends on 8) The non-transitory machine-readable medium of claim 8, wherein the dynamic pitch angle is searched using a binary search algorithm until the calculated distance from the ADV to the object is within a second predetermined threshold of the perceived distance from the ADV to the object. ### Claim 14 (depends on 8) The non-transitory machine-readable medium of claim 8, wherein the dynamic pitch angle is searched using a greedy search algorithm until the calculated distance from the ADV to the object is within a second predetermined threshold of the perceived distance from the ADV to the object. ### Claim 15 A data processing system, comprising: one or more processors; and a memory coupled to the one or more processors to store instructions, which when executed by the one or more processors, cause the one or more processors to perform operations, the operations including perceiving an object in a perspective view of the ADV using a plurality of sensors mounted on the ADV including capturing an image of the object using a monocular image capturing device; determining a two-dimensional (2D) bounding box for the object and a perceived distance from the ADV to the object based on the image of the object; searching, within an image space of the image of the object, for a dynamic pitch angle based on the 2D bounding box and the perceived distance from the ADV to the object by modifying at least a previous extrinsic calibration value to calculate a distance value; determining a distance from the ADV to a point on a ground plane of the perspective view of the ADV based on the determined dynamic pitch angle; and generating a driving trajectory based on the determined distance for the point on the ground plane to control the ADV. ### Claim 16 (depends on 15) The system of claim 15, wherein modifying at least a previous extrinsic calibration value includes modifying a pitch angle associated with a previous extrinsic calibration value for the monocular image capturing device. ### Claim 17 (depends on 15) The system of claim 15, wherein the 2D bounding box for the object is determined by: applying a machine learning algorithm model to the image capturing the object in the perspective view; and determining a dimension of the 2D bounding box bounding the object based on the machine learning algorithm model. ### Claim 18 (depends on 15) The system of claim 15, further comprises determining a ground point of contact of the 2D bounding box to calculate a distance from the ADV to the object based on the ground point of contact for the 2D bounding box. ### Claim 19 (depends on 15) The system of claim 15, wherein searching for the dynamic pitch angle comprises calculating the distance from the ADV to the object based on formula D=h/tan(alpha+beta), where beta is the dynamic pitch angle to be calculated, D is the perceived distance, alpha is a previous calibration angle from a camera optical axis to the ground, and h is an estimated height of the object. ### Claim 20 (depends on 15) The system of claim 15, wherein the dynamic pitch angle is searched using a binary search algorithm until the calculated distance from the ADV to the object is within a second predetermined threshold of the perceived distance from the ADV to the object. ### Claim 21 (depends on 15) The system of claim 15, wherein the dynamic pitch angle is searched using a greedy search algorithm until the calculated distance from the ADV to the object is within a second predetermined threshold of the perceived distance from the ADV to the object.