# US 2021/0279903 A1 · Distance measuring camera > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0279903 A1 - Document ID: US20210279903A1 - Status: Published application - Kind: A1 - Application number: 17/260,196 - Filed: 2019-07-08 - Published: 2021-09-09 - Earliest claimed: 2018-07-18 - Assignees: Mitsumi Electric Co., Ltd. - Applicants: Mitsumi Electric Co., Ltd. - Inventors: Satoru SUTO - Web page: https://patentgrep.com/patent/US20210279903A1 ## CPC - G06T 7/593: from stereo images - G01C 3/08: Use of electric radiation detectors - H04N 13/239: using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence) - G06T 2207/10012: Stereo images - G06T 2207/10028: Range image; Depth image; 3D point clouds - H04N 2013/0081: {Depth or disparity estimation from stereoscopic image signals} - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A distance measuring camera contains a first imaging system for obtaining a first image containing a first subject image, a second imaging system for obtaining a second image containing a second subject image, a size obtaining part for detecting a plurality of feature points of the first subject image in the first image and measure a distance between the feature points of the first subject image to obtain a size of the first subject image and utilizes an epipolar line to detect a plurality of feature points of the second subject image respectively corresponding to the plurality of feature points of the first subject image and measures a distance between the feature points of the second subject image to obtain a size of the second subject image and a distance calculating part for calculating a distance to the subject based an image magnification ratio between a magnification of the first subject image and a magnification of the first subject image and a magnification of the second subject image. ## Claims ### Claim 1 A distance measuring camera, comprising: a first imaging system including a first optical system for collecting light from a subject to form a first subject image and a first image sensor for imaging the first subject image to obtain a first image containing the first subject image; a second imaging system including a second optical system for collecting the light from the subject to form a second subject image and arranged so as to be shifted from the first optical system in a direction perpendicular to an optical axis direction of the first optical system and a second image sensor for imaging the second subject image to obtain a second image containing the second subject image; a size obtaining part for detecting a plurality of feature points of the first subject image in the first image and measuring a distance between the plurality of feature points of the first subject image to obtain a size of the first subject image and detecting a plurality of feature points of the second subject image in the second image respectively corresponding to the plurality of feature points of the first subject image and measuring a distance between the plurality of feature points of the second subject image to obtain a size of the second subject image; and a distance calculating part for calculating a distance to the subject based on an image magnification ratio between a magnification of the first subject image and a magnification of the second subject image which is obtained as a ratio between the size of the first subject image and the size of the second subject image obtained by the size obtaining part, wherein the size obtaining part detects the plurality of feature points of the second subject image in the second image by searching pixels on a plurality of epipolar lines in the second image respectively corresponding to the plurality of feature points of the first subject image. ### Claim 2 (depends on 1) The distance measuring camera as claimed in claim 1, wherein the size obtaining part derives the plurality of epipolar lines in the second image respectively corresponding to the plurality of feature points of the first subject image based on a model in which characteristics and arrangements of the first imaging system and the second imaging system are taken into consideration. ### Claim 3 (depends on 2) The distance measuring camera as claimed in claim 2, wherein each of the epipolar lines in the second image respectively corresponding to the plurality of feature points of the first subject image is represented by the following equation (1). yv 2 = ( G 1 ⁢ G 6 - G 2 ⁢ G 5 ) ⁢ xu 2 + G 3 ⁢ G 6 - G 4 ⁢ G 5 G 1 ⁢ G 4 - G 2 ⁢ G 3 ⁢ ⁢ where ⁢ ⁢ { G 1 = EP 2 ⁢ L 1 ⁡ ( R 32 ⁢ yv 1 + R 31 ⁢ xu 1 ) + K 1 ⁡ ( EP 2 ⁢ D + L 2 ) G 2 = EP 1 ⁢ EP 2 ⁡ ( R 32 ⁢ yv 1 + R 31 ⁢ xu 1 ) + EP 2 ⁢ K 1 ⁢ R 33 G 3 = - K 2 ⁢ L 1 ⁡ ( R 12 ⁢ yv 1 + R 11 ⁢ xu 1 ) + K 1 ⁢ K 2 ⁢ P x G 4 = - K 2 ⁢ EP 1 ⁡ ( R 12 ⁢ yv 1 + R 11 ⁢ xu 1 ) - K 1 ⁢ K 2 ⁢ R 13 G 5 = - K 2 ⁢ L 1 ⁡ ( R 22 ⁢ yv 1 + R 21 ⁢ xu 1 ) + K 1 ⁢ K 2 ⁢ P y G 6 = - K 2 ⁢ EP 1 ⁡ ( R 22 ⁢ yv 1 + R 21 ⁢ xu 1 ) - K 1 ⁢ K 2 ⁢ R 23 xu 1 = x 1 - u 1 yv 1 = y 1 - v 1 xu 2 = x 2 - u 2 yv 2 = y 2 - v 2 K 1 = f 1 · ( f 1 2 - EP 1 · f 1 + EP 1 · a FD ⁢ ⁢ 1 ) PS 1 · ( a FD ⁢ ⁢ 1 - f 1 ) K 2 = f 2 · ( f 2 2 - EP 2 · f 2 + EP 2 · a FD ⁢ ⁢ 2 ) PS 2 · ( a FD ⁢ ⁢ 2 - f 2 ) L 1 = f 1 2 - EP 1 · f 1 L 2 = f 2 2 - EP 2 · f 2 ( 1 ) where “x1”and “y1” are respectively x and y coordinates of arbitrary one of the plurality of feature points of the first subject image in the first image, “x2” and “y2” are respectively x and y coordinates of a feature point of the second subject image in the second image corresponding to the arbitrary one of the plurality of feature points of the first subject image, “Px” and “Py” are respectively values in x and y directions of a parallel disparity between a front principal point of the first optical system and a front principal point of the second optical system, “D” is a depth disparity in the optical axis direction of the first optical system or the second optical system between the first optical system and the second optical system, “PS1” is a pixel size of the first image sensor, “PS2” is a pixel size of the second image sensor, “f1” is a focal length of the first optical system, “f2” is a focal length of the second optical system, “EP1” is a distance from an exit pupil of the first optical system to an image formation position of the first subject image when the subject is located at an infinite distance point, “EP2” is a distance from an exit pupil of the second optical system to an image formation position of the second subject image when the subject is located at the infinite distance point, “aFD1” is a distance from the front principal point of the first optical system to the subject when the first subject image is in best focus on an imaging surface of the first image sensor, “aFD2” is a distance from the front principal point of the second optical system to the subject when the second subject image is in the best focus on an imaging surface of the second image sensor, “u1” and “v1” are respectively x and y coordinates of an optical axis of the first optical system in the first image, “u2” and “v2” are respectively x and y coordinates of an optical axis of the second optical system in the second image, and “R11”, “R12”, “R13”, “R21”, “R22”, “R23”, “R31”, “R32” and “R33” are components of a rotation matrix of the second imaging system with respect to the first imaging system. ### Claim 4 (depends on 1) The distance measuring camera as claimed in claim 1, wherein the first optical system and the second optical system are configured so that a change of the magnification of the first subject image according to the distance to the subject is different from a change of the magnification of the second subject image according to the distance to the subject. ### Claim 5 (depends on 4) The distance measuring camera as claimed in claim 4, wherein the first optical system and the second optical system are configured so that a focal length of the first optical system and a focal length of the second optical system are different from each other, and thereby the change of the magnification of the first subject image according to the distance to the subject is different from the change of the magnification of the second subject image according to the distance to the subject. ### Claim 6 (depends on 4) The distance measuring camera as claimed in claim 4, wherein the first optical system and the second optical system are configured so that a distance from an exit pupil of the first optical system to an image formation position of the first subject image formed by the first optical system when the subject is located at an infinite distance point is different from a distance from an exit pupil of the second optical system to an image formation position of the second subject image formed by the second optical system when the subject is located at the infinite distance point, and thereby the change of the magnification of the first subject image according to the distance to the subject is different from the change of the magnification of the second subject image according to the distance to the subject. ### Claim 7 (depends on 1) The distance measuring camera as claimed in claim 1, wherein a depth disparity in the optical axis direction of the first optical system or the second optical system exists between a front principal point of the first optical system and a front principal point of the second optical system, and thereby the change of the magnification of the first subject image according to the distance to the subject is different from the change of the magnification of the second subject image according to the distance to the subject.