US 11,590,655 B2 · External parameter calibration method for robot sensors and apparatus and robot with the same

An external parameter calibration method for sensors of a robot, comprising:

obtaining first sensor data and second sensor data obtained through a first sensor and a second sensor of the robot by collecting position information of a calibration reference object;
converting the first sensor data and the second sensor data to a same coordinate system to obtain corresponding first converted sensor data and second converted sensor data;
determining a first coordinate of a reference point of the calibration reference object based on the first converted sensor data, and determining a second coordinate of the reference point of the calibration reference object based on the second converted sensor data, and using the first coordinate and the second coordinate as a set of coordinate data;
returning to the step of obtaining the first sensor data and the second sensor data obtained through the first sensor and the second sensor of the robot by collecting the position information of the calibration reference object in response to a relative positional relationship between the robot and the calibration reference object being changed until N sets of the coordinate data are obtained, wherein N is a positive integer not less than an amount of unknown degrees of freedom in an external parameter;
and
calculating the external parameter between the first sensor and the second sensor based on the N sets of the coordinate data, wherein the external parameter is a relative positional relationship parameter between the first sensor and the second sensor;
wherein the first sensor is a single-line radar, the second sensor is a visual sensor, and the step of obtaining the first sensor data and the second sensor data obtained through the first sensor and the second sensor of the robot by collecting the position information of the calibration reference object comprises:
obtaining visual sensor data obtained through the visual sensor by collecting the position information of the calibration reference object;
and
extracting corresponding row data from the visual sensor data to take as the second sensor data based on a measured height of the single-line radar, in response to a height of the single-line radar being within a measurement range of the vision sensor;
and
selecting data with a closest height from the visual sensor data to take as the second sensor data, in response to the height of the single-line radar not being within the measurement range of the vision sensor;
wherein the calculating the external parameter between the first sensor and the second sensor based on the N sets of the coordinate data, wherein the external parameter is the relative positional relationship parameter between the first sensor and the second sensor comprises:
obtaining a conversion relationship between positions of the first sensor and the second sensor;
and
calculating the relative positional relationship parameter between the first sensor and the second sensor based on the N sets of the coordinate data and the conversion relationship;
and
wherein the conversion relationship is represented by a parameter to be solved, a data set of the first sensor is (xi,yi) and a data set of the second sensor is (xi′,yi′), and for a 2D navigation application of the robot, the parameter to be solved is (Δx, Δy, Δyaw), the conversion relationship is expressed as following matrix equation:
[
x
i
y
i
1
]
=
[
cos
⁡
(
Δ
⁢
⁢
yaw
)
sin
⁡
(
Δ
⁢
⁢
yaw
)
Δ
⁢
⁢
x
-
sin
⁡
(
Δ
⁢
⁢
yaw
)
cos
⁡
(
Δ
⁢
⁢
yaw
)
Δ
⁢
⁢
y
0
0
1
]
⁡
[
x
i
′
y
i
′
1
]
;
the matrix equation is solved to obtain;
[
x
i
y
i
1
]
=
[
x
i
′
⁢
cos
⁡
(
Δ
⁢
⁢
yaw
)
+
y
i
′
*
sin
⁡
(
Δ
⁢
⁢
yaw
)
+
Δ
⁢
⁢
x
-
x
i
′
⁢
sin
⁡
(
Δ
⁢
⁢
yaw
)
+
y
i
′
*
cos
⁡
(
Δ
⁢
⁢
yaw
)
+
Δ
⁢
⁢
y
1
]
;
the solved matrix equation is converted to obtain;
[
x
i
y
i
1
]
=
[
x
i
′
y
i
′
1
0
y
i
′
-
x
i
′
0
1
]
⁡
[
cos
⁡
(
Δ
⁢
⁢
yaw
)
sin
⁡
(
Δ
⁢
⁢
yaw
)
Δ
⁢
⁢
x
Δ
⁢
⁢
y
]
;
the obtained matrix equation is described as a problem of solving a linear equation of Ax=b:
A
=
[
x
0
′
y
0
′
1
0
y
0
′
-
x
0
′
0
1
⋮
x
n
′
y
n
′
1
0
y
n
′
-
x
n
′
0
1
]
2
⁢
n
*
4
;
b
=
[
x
0
y
0
⋮
x
n
y
n
]
2
⁢
n
*
1
;
and
the following formula is obtained:
[
cos
⁡
(
Δ
⁢
yaw
)
sin
⁡
(
Δ
⁢
yaw
)
Δ
⁢
x
Δ
⁢
y
]
=
[
x
0
′
y
0
′
1
0
y
0
′
-
x
0
′
0
1
·
·
·
x
n
′
y
n
′
1
0
y
n
′
-
x
n
′
0
1
]
2
⁢
n
*
4
-
1
[
x
0
y
0
·
·
·
x
n
y
n
]
2
⁢
n
*
1
;
⁢
Δ
⁢
yaw
=
arctan
⁡
(
sin
⁡
(
Δ
⁢
yaw
)
cos
⁡
(
Δ
⁢
yaw
)
)
;
each value in the external parameter (Δx, Δy, Δyaw) is solved, where n represents n sets of coordinate data.

390 more elements

An external parameter calibration apparatus for sensors of a robot, comprising:

an obtaining module configured to obtain first sensor data and second sensor data obtained through a first sensor and a second sensor of the robot by collecting position information of a calibration reference object;
a conversion module configured to convert the first sensor data and the second sensor data to a same coordinate system to obtain corresponding first converted sensor data and second converted sensor data;
a determination module configured to determine a first coordinate of a reference point of the calibration reference object based on the first converted sensor data, and determining a second coordinate of the reference point of the calibration reference object based on the second converted sensor data, and using the first coordinate and the second coordinate as a set of coordinate data;
a loop module configured to return to the obtaining the first sensor data and the second sensor data obtained through the first sensor and the second sensor of the robot by collecting the position information of the calibration reference object in response to a relative positional relationship between the robot and the calibration reference object being changed until N sets of the coordinate data are obtained, wherein N is a positive integer not less than an amount of unknown degrees of freedom in an external parameter;
and
a calculation module configured to calculate the external parameter between the first sensor and the second sensor based on the N sets of the coordinate data, wherein the external parameter is a relative positional relationship parameter between the first sensor and the second sensor;
wherein the first sensor is a single-line radar, the second sensor is a visual sensor, and the obtaining module is further configured to:
obtain visual sensor data obtained through the visual sensor by collecting the position information of the calibration reference object;
and
extract corresponding row data from the visual sensor data to take as the second sensor data based on a measured height of the single-line radar, in response to a height of the single-line radar being within a measurement range of the vision sensor;
and
select data with a closest height from the visual sensor data to take as the second sensor data, in response to the height of the single-line radar not being within the measurement range of the vision sensor;
wherein the calculation module is further configured to:
obtain a conversion relationship between positions of the first sensor and the second sensor;
and
calculate the relative positional relationship parameter between the first sensor and the second sensor based on the N sets of the coordinate data and the conversion relationship;
and
wherein the conversion relationship is represented by a parameter to be solved, a data set of the first sensor is (xi,yi) and a data set of the second sensor is (xi′,yi′), and for a 2D navigation application of the robot, the parameter to be solved is (Δx, Δy, Δyaw), the conversion relationship is expressed as following matrix equation;
[
x
i
y
i
1
]
=
[
cos
⁡
(
Δ
⁢
⁢
yaw
)
sin
⁡
(
Δ
⁢
⁢
yaw
)
Δ
⁢
⁢
x
-
sin
⁡
(
Δ
⁢
⁢
yaw
)
cos
⁡
(
Δ
⁢
⁢
yaw
)
Δ
⁢
⁢
y
0
0
1
]
⁡
[
x
i
′
y
i
′
1
]
;
the matrix equation is solved to obtain;
[
x
i
y
i
1
]
=
[
x
i
′
⁢
cos
⁡
(
Δ
⁢
⁢
yaw
)
+
y
i
′
*
sin
⁡
(
Δ
⁢
⁢
yaw
)
+
Δ
⁢
⁢
x
-
x
i
′
⁢
sin
⁡
(
Δ
⁢
⁢
yaw
)
+
y
i
′
*
cos
⁡
(
Δ
⁢
⁢
yaw
)
+
Δ
⁢
⁢
y
1
]
;
the solved matrix equation is converted to obtain;
[
x
i
y
i
1
]
=
[
x
i
′
y
i
′
1
0
y
i
′
-
x
i
′
0
1
]
⁡
[
cos
⁡
(
Δ
⁢
⁢
yaw
)
sin
⁡
(
Δ
⁢
⁢
yaw
)
Δ
⁢
⁢
x
Δ
⁢
⁢
y
]
;
the obtained matrix equation is described as a problem of solving a linear equation of Ax=b;
A
=
[
x
0
′
y
0
′
1
0
y
0
′
-
x
0
′
0
1
⋮
x
n
′
y
n
′
1
0
y
n
′
-
x
n
′
0
1
]
2
⁢
n
*
4
;
b
=
[
x
0
y
0
⋮
x
n
y
n
]
2
⁢
n
*
1
;
and
the following formula is obtained:
[
cos
⁡
(
Δ
⁢
yaw
)
sin
⁡
(
Δ
⁢
yaw
)
Δ
⁢
x
Δ
⁢
y
]
=
[
x
0
′
y
0
′
1
0
y
0
′
-
x
0
′
0
1
·
·
·
x
n
′
y
n
′
1
0
y
n
′
-
x
n
′
0
1
]
2
⁢
n
*
4
-
1
[
x
0
y
0
·
·
·
x
n
y
n
]
2
⁢
n
*
1
;
⁢
Δ
⁢
yaw
=
arctan
⁡
(
sin
⁡
(
Δ
⁢
yaw
)
cos
⁡
(
Δ
⁢
yaw
)
)
;
each value in the external parameter (Δx, Δy, Δyaw) is solved, where n represents a sets of coordinate data.

390 more elements

A robot comprising a non-transitory storage and a processor, wherein the non-transitory storage stores a computer program executable on the processor and the computer program comprises:

instructions for obtaining first sensor data and second sensor data obtained through a first sensor and a second sensor of the robot by collecting position information of a calibration reference object;
instructions for converting the first sensor data and the second sensor data to a same coordinate system to obtain corresponding first converted sensor data and second converted sensor data;
instructions for determining a first coordinate of a reference point of the calibration reference object based on the first converted sensor data, and determining a second coordinate of the reference point of the calibration reference object based on the second converted sensor data, and using the first coordinate and the second coordinate as a set of coordinate data;
instructions for returning to execute the instructions for obtaining the first sensor data and the second sensor data obtained through the first sensor and the second sensor of the robot by collecting the position information of the calibration reference object in response to a relative positional relationship between the robot and the calibration reference object being changed until N sets of the coordinate data re obtained, wherein N is a positive integer not less than an amount of unknown degrees of freedom in an external parameter;
and
instructions for calculating the external parameter between the first sensor and the second sensor based on the N sets of the coordinate data, wherein the external parameter is a relative positional relationship parameter between the first sensor and the second sensor;
wherein the first sensor is a single-line radar, the second sensor is a visual sensor, and the instructions for obtaining the first sensor data and the second sensor data obtained through the first sensor and the second sensor of the robot by collecting the position information of the calibration reference object comprises:
instructions for obtaining visual sensor data obtained through the visual sensor by collecting the position information of the calibration reference object;
and
instructions for extracting corresponding row data from the visual sensor data to take as the second sensor data based on a measured height of the single-line radar, in response to a height of the single-line radar being within a measurement range of the vision sensor;
and
instructions for selecting data with a closest height from the visual sensor data to take as the second sensor data, in response to the height of the single-line radar not being within the measurement range of the vision sensor;
wherein the instructions for calculating the external parameter between the first sensor and the second sensor based on the N sets of the coordinate data, wherein the external parameter is the relative positional relationship parameter between the first sensor and the second sensor comprise:
instructions for obtaining, a conversion relationship between positions of the first sensor and the second sensor;
and
instructions for calculating the relative positional relationship parameter between the first sensor and the second sensor based on the N sets of the coordinate data and the conversion relationship;
and
wherein the conversion relationship is represented by a parameter to:
be solved a data set of the first sensor is (xi,yi) and a data set of the second sensor is (xi′,yi′), and for a 2D navigation application of the robot, the parameter to be solved is (Δx, Δy, Δyaw), the conversion relationship is expressed as following matrix equation:
[
x
i
y
i
1
]
=
[
cos
⁡
(
Δ
⁢
⁢
yaw
)
sin
⁡
(
Δ
⁢
⁢
yaw
)
Δ
⁢
⁢
x
-
sin
⁡
(
Δ
⁢
⁢
yaw
)
cos
⁡
(
Δ
⁢
⁢
yaw
)
Δ
⁢
⁢
y
0
0
1
]
⁡
[
x
i
′
y
i
′
1
]
;
the matrix equation is solved to obtain;
[
x
i
y
i
1
]
=
[
x
i
′
⁢
cos
⁡
(
Δ
⁢
⁢
yaw
)
+
y
i
′
*
sin
⁡
(
Δ
⁢
⁢
yaw
)
+
Δ
⁢
⁢
x
-
x
i
′
⁢
sin
⁡
(
Δ
⁢
⁢
yaw
)
+
y
i
′
*
cos
⁡
(
Δ
⁢
⁢
yaw
)
+
Δ
⁢
⁢
y
1
]
;
the solved matrix equation is converted to obtain;
[
x
i
y
i
1
]
=
[
x
i
′
y
i
′
1
0
y
i
′
-
x
i
′
0
1
]
⁡
[
cos
⁡
(
Δ
⁢
⁢
yaw
)
sin
⁡
(
Δ
⁢
⁢
yaw
)
Δ
⁢
⁢
x
Δ
⁢
⁢
y
]
;
the obtained matrix equation is described as a problem of Solving a linear equation of Ax=b;
A
=
[
x
0
′
y
0
′
1
0
y
0
′
-
x
0
′
0
1
⋮
x
n
′
y
n
′
1
0
y
n
′
-
x
n
′
0
1
]
2
⁢
n
*
4
;
b
=
[
x
0
y
0
⋮
x
n
y
n
]
2
⁢
n
*
1
;
and
the following formula is obtained;
[
cos
⁡
(
Δ
⁢
yaw
)
sin
⁡
(
Δ
⁢
yaw
)
Δ
⁢
x
Δ
⁢
y
]
=
[
x
0
′
y
0
′
1
0
y
0
′
-
x
0
′
0
1
·
·
·
x
n
′
y
n
′
1
0
y
n
′
-
x
n
′
0
1
]
2
⁢
n
*
4
-
1
[
x
0
y
0
·
·
·
x
n
y
n
]
2
⁢
n
*
1
;
⁢
Δ
⁢
yaw
=
arctan
⁡
(
sin
⁡
(
Δ
⁢
yaw
)
cos
⁡
(
Δ
⁢
yaw
)
)
;
each value in the external parameter (Δx, Δy, Δyaw) is solved, where n represents n sets of coordinate data.

391 more elements

External parameter calibration method for robot sensors and apparatus and robot with the same

Life of this patent

External parameter calibration method for robot sensors and apparatus and robot with the same

Life of this patent

Abstract

Claims17