US 12,259,462 B2Granted

Ranging method, ranging device and ranging system

AssigneeHangzhou Hikvision Digital Technology Co., Ltd.

InventorsYushi Zhu, Jie Sun

Life of this patent

Estimating the term…

Loading patent…

Abstract

The present disclosure provides a ranging method, a ranging device, and a ranging system. The method includes: acquiring a video image and a radar signal set that are collected respectively by a camera and radar at the same time; determining position information corresponding to each target object in the video image; determining one or more target object that have matched a radar signal in the video image; determining a ranging algorithm based on the position information corresponding to the target objects that have matched radar signals and the radar signals matching the target objects; and calculating a distance between a target object that does not match a radar signal in the video image and a designated vehicle according to position information corresponding to the target object that does not match a radar signal in the video image and through the ranging algorithm.

Claims17

Dependent claims

A method, comprising:

acquiring a video image and a radar signal set that are collected respectively by a camera and a radar at a same time;
determining position information corresponding to each target object in the video image;
determining one or more target objects that have matched radar signals in the video image;
determining a ranging algorithm based on position information corresponding to the one or more target objects that have matched radar signals and the radar signals matching the one or more target objects;
and
calculating a distance between a target object having a mismatched radar signal in the video image and a designated vehicle according to position information corresponding to the target object having the mismatched radar signal in the video image and through the ranging algorithm,
wherein determining the one or more target objects that have matched radar signals in the video image comprises:
in response to determining that a video area of each target object of the one or more target objects in the video image overlaps with a projection area of a radar target in the video image, determining that the target object matches a radar signal corresponding to the radar target,
wherein the ranging algorithm comprises a monocular ranging model with a formula:
D
v
=
(
y
r
⁢
c
⁢
c
⁢
f
y
+
y
p
⁢
r
⁢
c
⁢
f
y
cos
⁢
θ
r
⁢
c
⁢
c
+
c
y
⁢
z
r
⁢
c
⁢
c
-
p
⁢
z
r
⁢
c
⁢
c
)
p
⁢
cos
⁡
(
θ
r
⁢
c
⁢
c
+
θ
p
⁢
p
)
-
f
y
⁢
sin
⁡
(
θ
r
⁢
c
⁢
c
+
θ
p
⁢
p
-
c
y
⁢
cos
⁡
(
θ
r
⁢
c
⁢
c
+
θ
p
⁢
p
)
)
,
where Dv denotes distance information, fy denotes a scaling factor of the camera in a v-axis direction of an image coordinate system, cy denotes a coordinate value of a principal point of the camera in a v-axis direction of the image coordinate system, θrcc denotes a reference pitch angle of the radar relative to the camera in a camera coordinate system, yrcc denotes a displacement relative to the camera in a Y-axis direction of the camera coordinate system, zrcc denotes a displacement relative to the camera in a Z-axis direction of the camera coordinate system, yprc denotes a displacement of a road relative to the radar in the Y-axis direction of the camera coordinate system, θpp denotes a pitch angle of the road, and p denotes corresponding position information of the target object in the video image.

114 more elements

A system, comprising:

a radar configured to collect radar signal sets;
a camera configured to collect a video image;
at least one processor;
and
at least one storage medium configured to store a computer program for execution by the at least one processor to perform operations comprising:
acquiring a video image and a radar signal set that are collected respectively by the camera and the radar at a same time;
determining position information corresponding to each target object in the video image;
determining one or more target objects that have matched radar signals in the video image;
determining a ranging algorithm based on position information corresponding to the one or more target objects that have matched radar signals and the radar signals matching the one or more target objects;
and
calculating a distance between a target object having a mismatched radar signal in the video image and a designated vehicle according to position information corresponding to the target object having the mismatched radar signal in the video image and through the ranging algorithm,
wherein determining one or more target objects that have matched radar signals in the video image comprises:
in response to determining that a video area of each target object of the one or more target objects in the video image overlaps with a projection area of a radar target in the video image, determining that the target object matches a radar signal corresponding to the radar target,
wherein the ranging algorithm comprises a monocular ranging model with a formula:
D
v
=
(
y
r
⁢
c
⁢
c
⁢
f
y
+
y
p
⁢
r
⁢
c
⁢
f
y
cos
⁢
θ
r
⁢
c
⁢
c
+
c
y
⁢
z
r
⁢
c
⁢
c
-
p
⁢
z
r
⁢
c
⁢
c
)
p
⁢
cos
⁡
(
θ
r
⁢
c
⁢
c
+
θ
p
⁢
p
)
-
f
y
⁢
sin
⁡
(
θ
r
⁢
c
⁢
c
+
θ
p
⁢
p
-
c
y
⁢
cos
⁡
(
θ
r
⁢
c
⁢
c
+
θ
p
⁢
p
)
)
,
where Dv denotes distance information, fy denotes a scaling factor of the camera in a v-axis direction of an image coordinate system, cy denotes a coordinate value of a principal point of the camera in a v-axis direction of the image coordinate system, θrcc denotes a reference pitch angle of the radar relative to the camera in a camera coordinate system, yrcc denotes a displacement relative to the camera in a Y-axis direction of the camera coordinate system, zrcc denotes a displacement relative to the camera in a Z-axis direction of the camera coordinate system, yprc denotes a displacement of a road relative to the radar in the Y-axis direction of the camera coordinate system, θpp denotes a pitch angle of the road, and p denotes corresponding position information of the target object in the video image.

119 more elements

A device, comprising:

at least one processor;
and
at least one non-transitory storage medium storing machine-executable instructions,
for execution by the at least one processor to perform operations comprising:
acquiring a video image and a radar signal set that are collected respectively by a camera and a radar at a same time;
determining position information corresponding to each target object in the video image;
determining one or more target objects that have matched radar signals in the video image;
determining a ranging algorithm based on position information corresponding to the one or more target objects that have matched radar signals and the radar signals matching the one or more target objects;
and
calculating a distance between a target object having a mismatched radar signal in the video image and a designated vehicle according to position information corresponding to the target object having the mismatched radar signal in the video image and through the ranging algorithm,
wherein determining one or more target objects that have matched radar signals in the video image comprises:
in response to determining that a video area of each target object of the one or more target objects in the video image overlaps with a projection area of a radar target in the video image, determining that the target object matches a radar signal corresponding to the radar target,
wherein the ranging algorithm comprises a monocular ranging model with a formula:
D
v
=
(
y
r
⁢
c
⁢
c
⁢
f
y
+
y
p
⁢
r
⁢
c
⁢
f
y
cos
⁢
θ
r
⁢
c
⁢
c
+
c
y
⁢
z
r
⁢
c
⁢
c
-
p
⁢
z
r
⁢
c
⁢
c
)
p
⁢
cos
⁡
(
θ
r
⁢
c
⁢
c
+
θ
p
⁢
p
)
-
f
y
⁢
sin
⁡
(
θ
r
⁢
c
⁢
c
+
θ
p
⁢
p
-
c
y
⁢
cos
⁡
(
θ
r
⁢
c
⁢
c
+
θ
p
⁢
p
)
)
,
where Dv denotes distance information, fy denotes a scaling factor of the camera in a v-axis direction of an image coordinate system, cy denotes a coordinate value of a principal point of the camera in a v-axis direction of the image coordinate system, θrcc denotes a reference pitch angle of the radar relative to the camera in a camera coordinate system, yrcc denotes a displacement relative to the camera in a Y-axis direction of the camera coordinate system, zrcc denotes a displacement relative to the camera in a Z-axis direction of the camera coordinate system, yprc denotes a displacement of a road relative to the radar in the Y-axis direction of the camera coordinate system, θpp denotes a pitch angle of the road, and p denotes corresponding position information of the target object in the video image.

118 more elements

Dependencies are inferred from the claim text.

Facts

Application: 17/251,599Filed 14 Jun 2019
Priority: CN 201810623786.015 Jun 2018
Published: US 2021/0270957 A12 Sep 2021
Granted: 25 Mar 2025
Term ends: Not estimatedThe source records don’t support an estimate.

Classification15 codes

G01S 13/867
Combinations of radar systems with non-radar systems, e.g. sonar, direction finder: combination of radar systems with cameras
G01C 3/12
Using a parallactic triangle with variable angles and a base of fixed length in the observation station, e.g. in the instrument: with monocular…
G01C 11/02
Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying: picture taking arrangements specially adapted for photogrammetry…
G01S 7/41
Of systems according to group G01S13/00: using analysis of echo signal for target characterisation; Target signature; Target cross-section