US 10,762,658 B2Granted

Method and image pick-up apparatus for calculating coordinates of object being captured using fisheye images

AssigneeAltek Corporation

InventorPo-Cheng Wu

Life of this patent

Estimating the term…

Loading patent…

Abstract

A method and an image pick-up apparatus for calculating coordinates of an object being captured by using fisheye images are provided. In the method, a first and a second fisheye image containing the object are respectively captured by using a first and a second fisheye lens. Coordinates of the object in the first and the second fisheye image are detected. A first and a second azimuth angle of the object relative to fisheye centers of the first and the second fisheye image on an image sensor plane are accordingly calculated. Distances between the coordinates of the object and the fisheye centers are respectively transformed into a first and a second incident angle through lens curves of the fisheye lenses. Three-dimensional coordinates of the object are calculated according to the first and the second azimuth angle, the first and the second incident angle, and a baseline distance between the fisheye lenses.

Claims16

Dependent claims

A method for calculating coordinates of an object being captured by using fisheye images, adapted for an image pick-up apparatus having a first fisheye lens and a second fisheye lens, wherein a baseline distance is provided between the first fisheye lens and the second fisheye lens, and the method comprises:

capturing a first fisheye image and a second fisheye image containing an object respectively by using the first fisheye lens and the second fisheye lens;
detecting first coordinates of the object in the first fisheye image and second coordinates of the object in the second fisheye image;
calculating a first azimuth angle and a second azimuth angle of the object relative to fisheye centers of the first fisheye image and the second fisheye image on an image sensor plane of the first fisheye lens and the second fisheye lens according to the first coordinates and the second coordinates;
transforming a first distance between the first coordinates and the fisheye center of the first fisheye image and a second distance between the second coordinates and the fisheye center of the second fisheye image into a first incident angle and a second incident angle respectively through lens curves of the first fisheye lens and the second fisheye lens;
and
calculating three-dimensional coordinates of the object by using a trigonometric function according to the first azimuth angle, the second azimuth angle, the first incident angle, the second incident angle, and the baseline distance,
wherein the first azimuth angle is φl, the second azimuth angle is φr, the first incident angle is θl, the second incident angle is θr, the baseline distance is B, and the three-dimensional coordinates of the object are (x, y, z), wherein
x
=
B
⁢
⁢
sin
⁢
⁢
φ
l
⁢
⁢
cos
⁢
⁢
φ
r
sin
⁡
(
φ
l
+
φ
r
)
;
y
=
B
⁢
⁢
sin
⁢
⁢
φ
l
⁢
⁢
sin
⁢
⁢
φ
r
sin
⁡
(
φ
l
+
φ
r
)
;
and
z
=
B
⁢
⁢
cos
⁢
⁢
θ
l
⁢
⁢
sin
⁢
⁢
φ
r
sin
⁢
⁢
θ
l
⁢
⁢
sin
⁡
(
φ
l
+
φ
r
)
⁢
⁢
or
⁢
⁢
z
=
B
⁢
⁢
cos
⁢
⁢
θ
r
⁢
⁢
sin
⁢
⁢
φ
l
sin
⁢
⁢
θ
r
⁢
⁢
sin
⁡
(
φ
l
+
φ
r
)
.

131 more elements

An image pick-up apparatus, comprising:

a first fisheye lens;
a second fisheye lens, a baseline distance being provided between the first fisheye lens and the second fisheye lens;
a storage device, storing a plurality of modules;
a processor, coupled to the first fisheye lens, the second fisheye lens, and the storage device and accessing and executing the modules stored in the storage device, the modules comprising:
an image pick-up module, capturing a first fisheye image and a second fisheye image containing an object respectively by using the first fisheye lens and the second fisheye lens;
an object detection module, detecting first coordinates and second coordinates of the object in the first fisheye image and the second fisheye image;
an azimuth angle calculation module, calculating a first azimuth angle and a second azimuth angle of the object relative to fisheye centers of the first fisheye image and the second fisheye image on an image sensor plane of the first fisheye lens and the second fisheye lens according to the first coordinates and the second coordinates;
an incident angle calculation module, transforming a first distance between the first coordinates and the fisheye center of the first fisheye image and a second distance between the second coordinates and the fisheye center of the second fisheye image into a first incident angle and a second incident angle respectively through lens curves of the first fisheye lens and the second fisheye lens;
and
a coordinate calculation module, calculating three-dimensional coordinates of the object by using a trigonometric function according to the first azimuth angle, the second azimuth angle, the first incident angle, the second incident angle, and the baseline distance,
wherein the first azimuth angle is φl, the second azimuth angle is φr, the first incident angle is θl, the second incident angle is θr, the baseline distance is B, and the three-dimensional coordinates of the object are (x, y, z), wherein
x
=
B
⁢
⁢
sin
⁢
⁢
φ
l
⁢
⁢
cos
⁢
⁢
φ
r
sin
⁡
(
φ
l
+
φ
r
)
;
y
=
B
⁢
⁢
sin
⁢
⁢
φ
l
⁢
⁢
sin
⁢
⁢
φ
r
sin
⁡
(
φ
l
+
φ
r
)
;
and
z
=
B
⁢
⁢
cos
⁢
⁢
θ
l
⁢
⁢
sin
⁢
⁢
φ
r
sin
⁢
⁢
θ
l
⁢
⁢
sin
⁡
(
φ
l
+
φ
r
)
⁢
⁢
or
⁢
⁢
z
=
B
⁢
⁢
cos
⁢
⁢
θ
r
⁢
⁢
sin
⁢
⁢
φ
l
sin
⁢
⁢
θ
r
⁢
⁢
sin
⁡
(
φ
l
+
φ
r
)
.

135 more elements

A method for calculating coordinates of an object captured by using fisheye images, adapted for an image pick-up apparatus having a fisheye lens, the method comprising:

capturing a fisheye image containing an object by using the fisheye lens;
detecting coordinates of the object in the fisheye image;
calculating an azimuth angle of the object relative to a fisheye center of the fisheye image on an image sensor plane of the fisheye lens according to the coordinates;
transforming a distance between the coordinates and the fisheye center of the fisheye image into an incident angle through a lens curve of the fisheye lens;
calculating a distance between the object and the fisheye lens according to a width of the object in the fisheye image in a direction facing the fisheye center;
and
calculating three-dimensional coordinates of the object by using a trigonometric function according to the azimuth angle, the incident angle, and the distance,
wherein the width of the object in the fisheye image is wT, an actual width of the object is WT, an image width of the fisheye image is WI, and a field of view (FOV) of the fisheye lens is F, and a distance r between the object and the fisheye lens is:
r
=
W
T
w
T
×
W
I
F
×
180
π
.

19 more elements

An image pick-up apparatus, comprising:

a fisheye lens;
a storage device, storing a plurality of modules;
a processor, coupled to the fisheye lens and the storage device and accessing and executing the modules stored in the storage device, the modules comprising:
an image pick-up module, capturing an image containing an object by using the fisheye lens;
an object detection module, detecting coordinates of the object in the fisheye image;
an azimuth angle calculation module, calculating an azimuth angle of the object relative to a fisheye center of the fisheye image on an image sensor plane of the fisheye lens according to the coordinates;
an incident angle calculation module, transforming a distance between the coordinates and the fisheye center of the fisheye image into an incident angle through a lens curve of the fisheye lens;
a distance calculation module, calculating a distance between the object and the fisheye lens according to a width of the object in the fisheye image in a direction facing the fisheye center;
and
a coordinate calculation module, calculating three-dimensional coordinates of the object by using a trigonometric function according to the azimuth angle, the incident angle, and the distance,
wherein the width of the object in the fisheye image is wT, an actual width of the object is WT, an image width of the fisheye image is WI, a field of view (FOV) of the fisheye lens is F, and a distance r between the object and the fisheye lens is:
r
=
W
T
w
T
×
W
I
F
×
180
π
.

22 more elements

Dependencies are inferred from the claim text.

Facts

Application: 16/164,810Filed 19 Oct 2018
Priority: TW 106136425 A24 Oct 2017
Published: US 2019/0122379 A125 Apr 2019
Granted: 1 Sep 2020
Term ends: Not estimatedThe source records don’t support an estimate.

Classification7 codes

G06T 7/62
Analysis of geometric attributes: of area, perimeter, diameter or volume
G06T 7/593
From multiple images: from stereo images
H04N 23/45
Cameras or camera modules comprising electronic image sensors; Control thereof: for generating image signals from two or more image sensors being…
G06T 2207/10012
Still image; Photographic image: stereo images