US 11,910,092 B2 · Panoramic look-around view generation method, in-vehicle device and in-vehicle system

Abstract

The current disclosure relates to a panoramic look-around view generation method, an in-vehicle device and an in-vehicle system. The method comprises the following steps of: acquiring images of areas around a vehicle, steering wheel angle information and vehicle speed information; transforming and mosaicking the images to generate a time-related look-around view, and using an ORB algorithm to extract characteristic points; using steering wheel angle information and vehicle speed information to calculate a positron of a characteristic point of the look-around view at previous time in the look-around view at current time according to a vehicle motion model, and selecting a characteristic point located near the position in the look-around view at current time to be matched with the characteristic point of the look-around view at previous time; calculating an affine transformation matrix and performing affine transformation, and performing weighted fusion with the look-around view at current time and storing it; repeating the above steps to obtain a continuously updated panoramic look-around view. The method, the device and the system can eliminate blind area in the underbody region, realize a panoramic perspective view, and have fast calculation and high accuracy.

A panoramic look-around view generation method, comprising steps of:

acquiring images of areas around a vehicle from a plurality of cameras installed on the vehicle, and acquiring steering wheel angle information and vehicle speed information from the vehicle;
generating a look-around view associated with a first time by transforming and mosaicking images acquired by the plurality of cameras, using the look-around view generated at a first time as the look-around view at the first time, and saving the look-around view at the first time;
(i) acquiring images of areas around the vehicle from a plurality of cameras installed on the vehicle, and acquiring steering wheel angle information and vehicle speed information from the vehicle;
(ii) generating the look-around view associated with a current time by transforming and mosaicking images acquired by the plurality of cameras, using the look-around view generated at the current time as the look-around view at the current time, and saving the look-around view at the current time;
(iii) performing a registration calculation on the look-around view at the current time and a saved look-around view at a previous time, wherein the registration calculation comprises following steps of:
(iv) respectively extracting a plurality of characteristic points of the look-around view at the previous time and a plurality of characteristic points of the look-around view at the current time by using an Oriented FAST and Rotated BRIEF (ORB) algorithm;
(v) calculating a position of a characteristic point of the look-around view at the previous time in the look-around view at the current time according to a vehicle motion model using the steering wheel angle information and the vehicle speed information;
selecting a characteristic point located near the position from the characteristic points of the look-around view at the current time to be matched with the characteristic point of the look-around view at the previous time;
and
(vi) calculating an affine transformation matrix between the look-around view at the previous time and the look-around view at the current time;
(vii) performing affine transformation on the look-around view at the previous time by using the affine transformation matrix and performing weighted fusion with the look-around view at the current time to generate an updated look-around view and save the updated look-around view;
repeating above steps (i)-(vii) to obtain continuously updated panoramic look-around views;
and
wherein a first transformation matrix of image coordinate systems is:
[
u
1
v
1
1
]
=
[
cos
⁢
⁢
θ
sin
⁢
⁢
θ
P
x
-
P
x
⁢
⁢
cos
⁢
⁢
θ
-
P
y
⁢
⁢
sin
⁢
⁢
θ
-
sin
⁢
⁢
θ
cos
⁢
⁢
θ
P
y
-
P
y
⁢
⁢
cos
⁢
⁢
θ
+
P
x
⁢
⁢
sin
⁢
⁢
θ
0
0
1
]
⁡
[
u
0
v
0
1
]
wherein (Px,Py) is the coordinate of a rotation center of the vehicle motion;
θ is a rotation angle of the vehicle from the previous time to the current time;
(u0,v0) is a coordinate of a characteristic point of the look-around view at the previous time in the look-around view at the previous time;
(u1,v1) is the coordinate of the characteristic point of the look-around view at the previous time in the look-around view at the current time.

93 more elements

Panoramic look-around view generation method, in-vehicle device and in-vehicle system

Life of this patent

Panoramic look-around view generation method, in-vehicle device and in-vehicle system

Life of this patent

Abstract

Claims4