US 10,567,736 B2Granted

Three-dimensional object detection apparatus, stereo camera apparatus, vehicle, and three-dimensional object detection method

AssigneeKYOCERA Corporation

InventorTomofumi Koishi

Life of this patent

Estimating the term…

Loading patent…

Abstract

A three-dimensional object detection apparatus includes an input interface and a controller. The input interface accepts input of first and second images having mutual parallax in a first direction. The controller generates a parallax image that uses a pixel value to represent a difference in the first direction between positions of a subject in the first image and in the second image and divides the parallax image into a plurality of partial regions extending along a second direction. For each partial region, the controller segments a distance in the direction towards the subject into a plurality of distance intervals; and calculates, on the basis of a pixel count of pixels belonging to the distance interval, an evaluation value of a three-dimensional object belonging to the distance interval. The controller uses a different threshold for each distance interval to determine whether or not a three-dimensional object is present.

Claims6

Dependent claims

A three-dimensional object detection apparatus comprising:

an input interface circuit;
and
a controller, wherein
the input interface is configured to accept input of a first image and a second image having mutual parallax in a first direction,
the controller is configured to
generate a parallax image that uses a pixel value to represent a difference in the first direction between a position of a subject in the first image and a position of the subject in the second image, and
divide at least a portion of the parallax image into a plurality of partial regions extending along a second direction intersecting the first direction,
the controller is configured, for each partial region, to
segment a distance in a direction towards the subject into a plurality of distance intervals,
calculate, from the pixel value, the distance interval to which each pixel of the partial region belongs,
calculate, on the basis of a pixel count of pixels belonging to the distance interval, an evaluation value corresponding to a length in the second direction of a three-dimensional object belonging to the distance interval, and
determine whether a three-dimensional object is present in each distance interval on the basis of the evaluation value,
the controller is configured to use a different threshold for each distance interval to determine that a three-dimensional object is present when the evaluation value is greater than the threshold,
the evaluation value is calculated by dividing the pixel count of each distance interval by a pixel count of the partial region in the first direction, and
the threshold satisfies a conditional expression,
f
·
(
tan
[
tan
-
1
(
h
0
Z
1
)
-
θ
0
]
-
tan
[
tan
-
1
(
h
0
Z
2
)
-
θ
0
]
)
⁢
/
⁢
P
<
Th
<
f
·
Y
min
Z
3
⁢
/
⁢
P
(
1
)
where Th is the threshold, f is a focal length of an optical system that captured the first image and the second image, θ0 is the angle of depression of the optical system that captured the first image and the second image, h0 is the attachment height of the optical system that captured the first image and the second image, Z1 is a near-side boundary and Z2 is a far-side boundary of a distance interval of a determination target, Z3 is a deemed distance to a predetermined three-dimensional object for the distance interval of the determination target, Ymin is a minimum height of a three-dimensional object to be detected, and P is a pixel pitch of an image.

68 more elements

A stereo camera apparatus comprising:

a stereo camera;
and
a controller, wherein
the stereo camera is configured to output a first image and a second image having mutual parallax in a first direction,
the controller is configured to
generate a parallax image that uses a pixel value to represent a difference in the first direction between a position of a subject in the first image and a position of the subject in the second image, and
divide at least a portion of the parallax image into a plurality of partial regions extending along a second direction intersecting the first direction,
the controller is configured, for each partial region, to
segment a distance in a direction towards the subject into a plurality of distance intervals,
calculate, from the pixel value, the distance interval to which each pixel of the partial region belongs,
calculate, on the basis of a pixel count of pixels belonging to the distance interval, an evaluation value corresponding to a length in the second direction of a three-dimensional object belonging to the distance interval, and
determine whether a three-dimensional object is present in each distance interval on the basis of the evaluation value,
the controller is configured to use a different threshold for each distance interval to determine that a three-dimensional object is present when the evaluation value is greater than the threshold,
the evaluation value is calculated by dividing the pixel count of each distance interval by a pixel count of the partial region in the first direction, and
the threshold satisfies a conditional expression,
f
·
(
tan
⁡
[
tan
-
1
⁡
(
h
0
Z
1
)
-
θ
0
]
-
tan
⁡
[
tan
-
1
⁡
(
h
0
Z
2
)
-
θ
0
]
)
/
P
<
T
⁢
⁢
h
<
f
·
Y
min
Z
3
/
P
(
1
)
where Th is the threshold, f is a focal length of an optical system that captured the first image and the second image, θ0 is the angle of depression of the optical system that captured the first image and the second image, h0 is the attachment height of the optical system that captured the first image and the second image, Z1 is a near-side boundary and Z2 is a far-side boundary of a distance interval of a determination target, Z3 is a deemed distance to a predetermined three-dimensional object for the distance interval of the determination target, Ymin is a minimum height of a three-dimensional object to be detected, and P is a pixel pitch of an image.

71 more elements

A vehicle comprising:

a stereo camera;
and
a controller, wherein
the stereo camera is configured to output a first image and a second image having mutual parallax in a first direction,
the controller is configured to
generate a parallax image that uses a pixel value to represent a difference in the first direction between a position of a subject in the first image and a position of the subject in the second image, and
divide at least a portion of the parallax image into a plurality of partial regions extending along a second direction intersecting the first direction,
the controller is configured, for each partial region, to
segment a distance in a direction towards the subject into a plurality of distance intervals,
calculate, from the pixel value, the distance interval to which each pixel of the partial region belongs,
calculate, on the basis of a pixel count of pixels belonging to the distance interval, an evaluation value corresponding to a length in the second direction of a three-dimensional object belonging to the distance interval, and
determine whether a three-dimensional object is present in each distance interval on the basis of the evaluation value,
the controller is configured to use a different threshold for each distance interval to determine that a three-dimensional object is present when the evaluation value is greater than the threshold,
the stereo camera is configured to image an external environment of the vehicle,
the evaluation value is calculated by dividing the pixel count of each distance interval by a pixel count of the partial region in the first direction, and
the threshold satisfies a conditional expression,
f
·
(
tan
⁡
[
tan
-
1
⁡
(
h
0
Z
1
)
-
θ
0
]
-
tan
⁡
[
tan
-
1
⁡
(
h
0
Z
2
)
-
θ
0
]
)
/
P
<
T
⁢
⁢
h
<
f
·
Y
min
Z
3
/
P
(
1
)
where Th is the threshold, f is a focal length of an optical system that captured the first image and the second image, θ0 is the angle of depression of the optical system that captured the first image and the second image, h0 is the attachment height of the optical system that captured the first image and the second image, Z1 is a near-side boundary and Z2 is a far-side boundary of a distance interval of a determination target, Z3 is a deemed distance to a predetermined three-dimensional object for the distance interval of the determination target, Ymin is a minimum height of a three-dimensional object to be detected, and P is a pixel pitch of an image.

72 more elements

A three-dimensional object detection method comprising:

accepting, from a stereo camera, input of a first image and a second image having mutual parallax in a first direction;
generating a parallax image that uses a pixel value to represent a difference in the first direction between a position of a subject in the first image and a position of the subject in the second image;
and
dividing at least a portion of the parallax image into a plurality of partial regions extending along a second direction intersecting the first direction;
for each partial region,
segmenting a distance in a direction towards the subject into a plurality of distance intervals;
calculating, from the pixel value, the distance interval to which each pixel of the partial region belongs;
and
calculating, on the basis of a pixel count of pixels belonging to the distance interval, an evaluation value corresponding to a length in the second direction of a three-dimensional object belonging to the distance interval;
and
using a different threshold for each distance interval, and determining that a three-dimensional object is present when the evaluation value is greater than the threshold, wherein
the evaluation value is calculated by dividing the pixel count of each distance interval by a pixel count of the partial region in the first direction, and
the threshold satisfies a conditional expression,
f
·
(
tan
⁡
[
tan
-
1
⁡
(
h
0
Z
1
)
-
θ
0
]
-
tan
⁡
[
tan
-
1
⁡
(
h
0
Z
2
)
-
θ
0
]
)
/
P
<
T
⁢
⁢
h
<
f
·
Y
min
Z
3
/
P
(
1
)
where Th is the threshold, f is a focal length of an optical system that captured the first image and the second image, θ0 is the angle of depression of the optical system that captured the first image and the second image, h0 is the attachment height of the optical system that captured the first image and the second image, Z1 is a near-side boundary and Z2 is a far-side boundary of a distance interval of a determination target, Z3 is a deemed distance to a predetermined three-dimensional object for the distance interval of the determination target, Ymin is a minimum height of a three-dimensional object to be detected, and P is a pixel pitch of an image.

69 more elements

Dependencies are inferred from the claim text.

Facts

Application: 15/739,481Filed 23 Jun 2016
Priority: JP 2015-12587523 Jun 2015
Published: US 2018/0324406 A18 Nov 2018
Granted: 18 Feb 2020
Term ends: Not estimatedThe source records don’t support an estimate.

Classification16 codes

H04N 13/225
Using a single two-dimensional [2D] image sensor: using parallax barriers
G01C 3/14
Using a parallactic triangle with variable angles and a base of fixed length in the observation station, e.g. in the instrument: with binocular…
G01C 11/06
Interpretation of pictures: by comparison of two or more pictures of the same area
G01S 11/12
Systems for determining distance or velocity not using reflection or reradiation: using electromagnetic waves other than radio waves