# US 10,769,813 B2 · Apparatus and method for calibrating surround-view camera systems > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,769,813 B2 - Document ID: US10769813B2 - Status: Granted - Kind: B2 - Application number: 16/114,706 - Filed: 2018-08-28 - Published: 2020-03-05 - Granted: 2020-09-08 - Earliest claimed: 2018-08-28 - Assignees: Bendix Commercial Vehicle Systems LLC - Applicants: Bendix Commercial Vehicle Systems, LLC - Inventors: Andreas U. Kuehnle; Hans M. Molin; Travis G. Ramler; Jeffrey R. Coleman; Cathy L. Boon; Zheng Li - Web page: https://patentgrep.com/patent/US10769813B2 ## CPC - G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration - G06T 7/85: {Stereo camera calibration} - G06T 7/55: from multiple images - H04N 13/282: for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems - H04N 13/351: for displaying simultaneously - H04N 17/002: {for television cameras} - G06T 2207/10012: Stereo images - G06T 2207/30208: Marker matrix - G06T 2207/30252: Vehicle exterior; Vicinity of vehicle - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A calibration apparatus and method are provided for a surround-view camera system having a plurality of cameras mounted to an object (e.g., a vehicle). A plurality of markers are arbitrarily placed around the object and configured to measure a plurality of distances between one another. The markers preferably include a rangefinder to measure distances. From at least one image from one or more cameras, a controller is configured to visually identify at least two markers which define a coordinate system. Based on the measured plurality of distances between markers and the visually identified markers, the controller generates calibration information as an output, preferably saved in the form of a look-up table, which will be retrieved and used when a display device generates a bird's-eye view for a viewer. Additionally, a marker that is part of the calibration apparatus is provided. ## Claims ### Claim 1 A calibration apparatus for a surround-view camera system having a plurality of cameras mounted to an object, comprising: a plurality of markers configured to measure a plurality of distances between at least a first one of the plurality of markers and at least two others of the plurality of markers; and a controller configured to visually identify the first marker and at least one of the at least two other markers from at least one image from the plurality of cameras, wherein the controller is configured to generate camera calibration information based on the plurality of measured distances and positions of the visually identified markers in the at least one image, wherein the controller visually identifies the first marker and the at least one of the at least two other markers via a visually distinctive characteristic on a corresponding marker, and the visually distinctive characteristic is one of a shape, a color, a texture, a size, a pattern or a reflectivity. ### Claim 2 (depends on 1) The calibration apparatus according to claim 1, wherein the corresponding marker or a portion of the corresponding marker is capable of being pivotably or tiltably adjusted such that the visually distinctive characteristic is readily observable from at least one of the plurality of cameras. ### Claim 3 (depends on 1) The calibration apparatus according to claim 1, wherein when a marker communicates with other markers or the controller, the marker includes a unique identification code such that the other markers or the controller are(is) able to identify which marker is communicating. ### Claim 4 (depends on 1) The calibration apparatus according to claim 1, wherein the at least one of the two other markers includes a second marker, the first marker defines an origin of a coordinate system and the second marker defines a first axis of the coordinate system, and the controller calculates coordinates within the coordinate system for one or more of the plurality of markers based on the measured plurality of distances. ### Claim 5 (depends on 4) The calibration apparatus according to claim 4, wherein a center of the first marker defines the origin of the coordinate system, and a line connecting between the center of the first marker and a center of the second marker is the first axis of the coordinate system. ### Claim 6 (depends on 4) The calibration apparatus according to claim 4, wherein the plurality of markers further include a third marker, and a coordinate for the third marker in the coordinate system is calculated based on a first measured distance between the first and third markers, and a second measured distance between the second and third markers. ### Claim 7 (depends on 6) The calibration apparatus according to claim 6, wherein the coordinate for the third marker in the coordinate system is one of two intersections between a first circle and a second circle, where the first circle has its origin at a center of the first marker with its radius being the first measured distance, and the second circle has its origin at a center of the second marker with its radius being the second measured distance. ### Claim 8 (depends on 7) The calibration apparatus according to claim 7, wherein, between the two intersections, the one that meets one or more predetermined conditions is determined to be the coordinate for the third marker. ### Claim 9 (depends on 7) The calibration apparatus according to claim 7, wherein the plurality of markers further include a fourth marker, and a coordinate for the fourth marker in the coordinate system is calculated based on a third measured distance between the first and fourth markers, and a fourth measured distance between the second and fourth markers. ### Claim 10 (depends on 9) The calibration apparatus according to claim 9, wherein the coordinate for the fourth marker in the coordinate system is one of two intersections between a third circle and a fourth circle, where the third circle has its origin at a center of the first marker with its radius being the third measured distance, and the fourth circle has its origin at a center of the second marker with its radius being the fourth measured distance. ### Claim 11 (depends on 10) The calibration apparatus according to claim 10, wherein, among four sets of candidate coordinates for the third and fourth markers based on the two intersections between the first and second circles, and the two intersections between the third and fourth circles, a set of candidate coordinates whose calculated distance therebetween matches a measured distance between the third and fourth markers is determined to be coordinates for the third and fourth markers. ### Claim 12 (depends on 1) The calibration apparatus according to claim 1, wherein the plurality of markers are deployed forming a grid, where each marker functions as a grid point. ### Claim 13 (depends on 1) The calibration apparatus according to claim 1, wherein each of the plurality of markers self-measures distances relative to other markers. ### Claim 14 (depends on 1) The calibration apparatus according to claim 1, wherein a hierarchy exists for the plurality of markers, where a marker functions as a master marker with respect to the rest of markers which are slave markers. ### Claim 15 (depends on 14) The calibration apparatus according to claim 14, wherein one or more of the slave markers transmit measured distances to the master marker or to the controller in the order as defined in the hierarchy. ### Claim 16 (depends on 1) The calibration apparatus according to claim 1, wherein one or more of the plurality of markers include(s) a rangefinder that is capable of measuring a distance between itself and its target. ### Claim 17 (depends on 1) The calibration apparatus according to claim 1, wherein one or more of the plurality of markers include(s) a body having one or more reflective surfaces that reflect a light pulse. ### Claim 18 (depends on 17) The calibration apparatus according to claim 17, wherein the body includes a stepped portion such that the body provides at least two reflective surfaces having different radii relative to a center line of the body. ### Claim 19 (depends on 18) The calibration apparatus according to claim 18, wherein a differential value of the different radii is uniquely associated with a specific marker such that the differential value is used to identify the specific marker. ### Claim 20 (depends on 19) The calibration apparatus according to claim 19, wherein the specific marker reflects a received light pulse from another marker at two or more reflective surfaces such that an identity of the specific marker is determined based on the reflected light pulses from the two or more reflective surfaces. ### Claim 21 (depends on 1) The calibration apparatus according to claim 1, wherein the generation of the camera calibration information includes the controller generating correlation information between a marker's location and a pixel of a corresponding camera based on the measured plurality of distances and the visually identified markers. ### Claim 22 (depends on 21) The calibration apparatus according to claim 21, wherein the controller generates homographic relationships among the plurality of cameras based on the correlation information. ### Claim 23 (depends on 22) The calibration apparatus according to claim 22, wherein the homographic relationships are saved in the form of a look-up table. ### Claim 24 (depends on 23) The calibration apparatus according to claim 23, further comprising: a surround view display device, wherein the display device is configured to display a continuous bird's-eye view image of at least a portion of a region around the object based on the camera calibration information and images from at least two of the plurality of cameras. ### Claim 25 (depends on 24) The calibration apparatus according to claim 24, wherein the continuous bird's-eye view image includes a 360° view of an entire region around the object. ### Claim 26 (depends on 24) The calibration apparatus according to claim 24, wherein the display device forms the continuous bird's-eye view image by stitching adjacent images using a homographic method. ### Claim 27 A method for calibrating a surround-view camera system having a plurality of cameras mounted to an object, the method comprising the steps of: placing a plurality of markers around the object; measuring a plurality of distances between at least a first one of the plurality of markers and at least two others of the plurality of markers; visually identifying the first marker and at least one of the at least two other markers from at least one image from the plurality of cameras; and generating camera calibration information based on the plurality of measured distances and positions of the visually identified markers in the at least one image, wherein the step of visually identifying the first marker and at least one of the at least two other markers is performed via a visually distinctive characteristic on a corresponding marker, and the visually distinctive characteristic is one of a shape, a color, a texture, a size, a pattern or a reflectivity. ### Claim 28 (depends on 27) The method according to claim 27, wherein the step of measuring a plurality of distances is performed via a rangefinder that is capable of measuring a distance between itself and its target.