# US 11,445,162 B2 · Method and device for calibrating binocular camera

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,445,162 B2
- Document ID: US11445162B2
- Status: Granted
- Kind: B2
- Application number: 17/123,999
- Filed: 2020-12-16
- Published: 2021-12-09
- Granted: 2022-09-13
- Earliest claimed: 2020-06-05
- Assignees: Beijing Smarter Eye Technology Co. Ltd.
- Applicants: Beijing Smarter Eye Technology Co. Ltd.
- Inventors: Zhao Sun; Haitao Zhu; Yongcai Liu; Shanshan Pei; Xinliang Wang
- Web page: https://patentgrep.com/patent/US11445162B2

## CPC

- H04N 13/246: Calibration of cameras
- B60R 11/04: Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
- G06T 7/73: using feature-based methods
- G06T 7/85: {Stereo camera calibration}
- H04N 13/239: using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence)
- G06T 2207/10012: Stereo images
- G06T 2207/20076: Probabilistic image processing
- G06T 2207/30252: Vehicle exterior; Vicinity of vehicle
- H04N 2013/0081: {Depth or disparity estimation from stereoscopic image signals}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

The present disclosure provides a method for calibrating a binocular camera, including: S2 of extracting feature points from an image set 1 and an image set 2 taken at two points separated from each other by a predetermined distance; S3 of fitting Gaussian distribution parameters of each feature point, and extracting a desired value as a theoretical disparity; S4 of selecting a common feature point, and calculating the predetermined quantity of frames at a first theoretical distance and the predetermined quantity of frames at a second theoretical distance; S5 of performing Gaussian fitting on a difference between the theoretical distances of the common feature point, and extracting a variance as an evaluation index; S6 of determining whether the evaluation index is smaller than a threshold, if yes, terminating the calibration, and otherwise, proceeding to S7; and S7 of adjusting posture parameters of the binocular camera, and returning to S3.

## Claims

### Claim 1

A method for calibrating a binocular camera, comprising:
S2 of extracting a plurality of feature points from an image set 1 and an image set 2 taken at two points separated from each other by a predetermined distance (s);
S3 of fitting Gaussian distribution parameters of each feature point, and extracting an expected value as a theoretical disparity;
S4 of selecting a common feature point amongst the plurality of feature points, and calculating a first theoretical distance (Z1) and a second theoretical distance (Z2) of the common feature point;
S5 of performing Gaussian fitting on a difference between the theoretical distances Z1 and Z2 of the common feature point (Δ=Z1−Z2), and extracting a variance (σ2) as an evaluation index;
S6 of determining whether the evaluation index is smaller than a threshold, terminating the calibration when the evaluation index is smaller than the threshold, or proceeding to S7 when the evaluation index is not smaller than the threshold; and
S7 of adjusting posture parameters of the binocular camera, and returning to S3,
wherein S3 comprises: S31 of calculating a disparity of each feature point with respect to each image in the image set 1; S32 of performing statistical analysis on the disparities in the predetermined quantify of frames with respect to each feature point to acquire a Gaussian fitting model; S33 of selecting the expected value of the Gaussian fitting model as the theoretical disparity of the feature point; S34 of repeating S31 to S33 with respect to each feature point in the image set 1 to acquire the theoretical disparities of all the feature points in the image set 1; and S35 of repeating S31 to S34 with respect to the image set 2 to acquire theoretical disparities of all the feature points in the image set 2.

### Claim 2 (depends on 1)

The method according to claim 1, further comprising: prior to S2, S1 of taking the image set 1 and the image set 2 at the two points separated from each other by the predetermined distance (s), wherein S1 comprises: S11 of taking the predetermined quantity of frames in a current scene in the case that the binocular camera is in a stationary state, and marking the frames as the image set 1; and S12 of moving the binocular camera from the predetermined distance (s) in a fixed direction in the same scene, taking the predetermined quantity of frames in the scene in the case that the binocular camera is in the stationary state, and marking the frames as the image set 2.

### Claim 3 (depends on 1)

The method according to claim 1, wherein S4 comprises: S41 of selecting the common feature points existing in the image set 1 and the image set 2; S42 of calculating the first theoretical distance (Z1) in accordance with the theoretical disparity of each selected common feature point in the image set 1; and S43 of calculating the second theoretical distance (Z2) in accordance with the theoretical disparity of each selected common feature point in the image set 2.

### Claim 4

A method for calibrating a binocular camera, comprising:
S1 of taking the image set 1 and the image set 2 at the two points separated from each other by a predetermined distance (s),
S2 of extracting a plurality of feature points from an image set 1 and an image set 2;
S3 of fitting Gaussian distribution parameters of each feature point, and extracting an expected value as a theoretical disparity;
S4 of selecting a common feature point, and calculating a first theoretical distance (Z1) and a second theoretical distance (Z2) of the common feature point;
S5 of obtaining an evaluation index, which comprises: S51 of calculating a difference between the first theoretical distance (Z1i) and the second theoretical distance (Z2i) of each selected common feature point according to a formula Δt=Z1i_31 Z2i; and S52 of performing the Gaussian fitting on the difference (Δi) between the theoretical distances of each selected common feature point to acquire Gaussian distribution with the expected value (Δc) being equal to the predetermined distance (s), and extracting the variance (σ2) as the evaluation index,
S6 of determining whether the evaluation index is smaller than a threshold, terminating the calibration when the evaluation index is smaller than the threshold, or proceeding to S7 when the evaluation index is not smaller than the threshold; and
S7 of adjusting posture parameters of the binocular camera, and returning to S3.

### Claim 5

A memory device storing therein a plurality of instructions, wherein the instructions are loaded and executed by a processor so as to perform the following steps:
S2 of extracting feature points from an image set 1 and an image set 2 taken at two points separated from each other by a predetermined distance (s);
S3 of fitting Gaussian distribution parameters of each feature point, and extracting an expected value as a theoretical disparity;
S4 of selecting a common feature point, and calculating a first theoretical distance (Z1) and a second theoretical distance (Z2) of the common feature point;
S5 of performing Gaussian fitting on a difference between the theoretical distances of the common feature point (Δ=Z1−Z2), and extracting a variance (σ2) as an evaluation index;
S6 of determining whether the evaluation index is smaller than a threshold, if yes, terminating the calibration, and otherwise, proceeding to S7; and
S7 of adjusting posture parameters of the binocular camera, and returning to S3,
wherein S3 comprises: S31 of calculating a disparity of each feature point with respect to each image in the image set 1; S32 of performing statistical analysis on the disparities in the predetermined quantity of frames with respect to each feature point to acquire a Gaussian fitting model; S33 of selecting the expected value of the Gaussian fitting model as the theoretical disparity of the feature point; S34 of repeating S31 to S33 with respect to each feature point in the image set 1 to acquire the theoretical disparities of all the feature points in the image set 1;
and S35 of repeating S31 to S34 with respect to the image set 2, to acquire theoretical disparities of all the feature points in the image set 2.

### Claim 6

A vehicle with a binocular camera, comprising a processor configured to execute a plurality of instructions, and a memory device storing therein the plurality of instructions, wherein the instructions are loaded and executed by the processor so as to perform the following steps:
S2 of extracting feature points from an image set 1 and an image set 2 taken at two points separated from each other by a predetermined distance (s);
S3 of fitting Gaussian distribution parameters of each feature point, and extracting an expected value as a theoretical disparity;
S4 of selecting a common feature point, and calculating a first theoretical distance (Z1) and a second theoretical distance (Z2) of the common feature point;
S5 of obtaining a variance (σ2) as an evaluation index, which comprises: S51 of calculating a difference between the first theoretical distance (Z1i) and the second theoretical distance (Z2i) of each selected common feature point through a formula Δi=Z1i−Z2i; and S52 of performing the Gaussian fitting on the difference (Δi) between the theoretical distances of each selected common feature point to acquire Gaussian distribution with the expected value (σc) being equal to the predetermined distance (s), and extracting the variance (σ2) as the evaluation index;
S6 of determining whether the evaluation index is smaller than a threshold, if yes, terminating the calibration, and otherwise, proceeding to S7; and
S7 of adjusting posture parameters of the binocular camera and returning to S3.

### Claim 7 (depends on 4)

The method of claim 4, wherein S1 comprises: S11 of taking the predetermined quantity of frames in a current scene in the case that the binocular camera is in a stationary state, and marking the frames as the image set 1; and S12 of moving the binocular camera from the predetermined distance (s) in a fixed direction in the same scene, taking the predetermined quantity of frames in the scene in the case that the binocular camera is in the stationary state, and marking the frames as the image set 2.