# US 2021/0331753 A1 · Stair climbing gait planning method and apparatus and robot using the same

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0331753 A1
- Document ID: US20210331753A1
- Status: Published application
- Kind: A1
- Application number: 16/885,227
- Filed: 2020-05-27
- Published: 2021-10-28
- Earliest claimed: 2020-04-24
- Assignees: Ubtech Robotics Corp Ltd
- Applicants: Ubtech Robotics Corp Ltd
- Inventors: JIE BAI; Ligang Ge; Hongge Wang; Yizhang Liu; Shuping Hu; Jianxin Pang; Youjun Xiong
- Web page: https://patentgrep.com/patent/US20210331753A1

## CPC

- B62D 57/02: with ground-engaging propulsion means, e.g. walking members
- B62D 57/032: with alternately or sequentially lifted supporting base and legs; with alternately or sequentially lifted feet or skid (B62D57/024 takes precedence)
- B25J 9/1669: {characterised by special application, e.g. multi-arm co-operation, assembly, grasping}
- B25J 9/1697: {Vision controlled systems}
- B25J 19/023: {including video camera means}
- B62D 57/024: specially adapted for moving on inclined or vertical surfaces (endless-track vehicles for ascending or descending stairs B62D55/075; hand-carts with provision for travelling up or down stairs B62B5/02)
- G05D 1/0234: {using optical markers or beacons (optical beacons per se G01S1/70)}
- G05D 1/0891: {specially adapted for land vehicles}
- G06F 17/18: for evaluating statistical data {, e.g. average values, frequency distributions, probability functions, regression analysis (forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04)}
- B62D: MOTOR VEHICLES; TRAILERS (steering, or guiding on a desired track, of agricultural machines or implements A01B69/00; wheels, castors, axles, increasing wheel adhesion B60B; vehicle tyres, tyre inflation or tyre changing B60C; connections between vehicles of a train or the like B60D; vehicles for use on rail and road, amphibious or convertible vehicles B60F; suspension arrangements B60G; heating, cooling, ventilating or other air treating devices B60H; windows, windscreens, non-fixed roofs, doors or similar devices, protective coverings for vehicles not in use B60J; propulsion plant arrangements, auxiliary drives, transmissions, controls, instrumentation or dashboards B60K; electric equipment or propulsion of electrically-propelled vehicles B60L; power supply for electrically-propelled vehicles B60M; passenger accommodation not otherwise provided for B60N; adaptations for load transportation or to carry special loads or objects B60P; arrangement of signalling or lighting devices, the mounting or supporting thereof or circuits therefor, for vehicles in general B60Q; vehicles, vehicle fittings or vehicle parts not otherwise provided for B60R; servicing, cleaning, repairing, supporting, lifting, or manoeuvring, not otherwise provided for, B60S; brake arrangements, brake control systems or parts thereof B60T; air-cushion vehicles B60V; motor-cycles, accessories therefor B62J, B62K; testing of vehicles G01M)
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

The present disclosure provides a stair climbing gait planning method and an apparatus and a robot using the same. The method includes: obtaining first visual measurement data through a visual sensor of the robot; converting the first visual measurement data to second visual measurement data; and performing a staged gait planning on a process of the robot to climb the staircase based on the second visual measurement data. Through the method, the visual measurement data is used as a reference to perform the staged gait planning on the process of the robot to climb the staircase, which greatly improves the adaptability of the robot in the complex scene of stair climbing.

## Claims

### Claim 1

A computer-implemented stair climbing gait planning method for a robot, comprising executing on a processor steps of:
obtaining first visual measurement data through a visual sensor of the robot, wherein the first visual measurement data is pose data of a preset marker on a step of a staircase in a body coordinate system of the robot;
converting the first visual measurement data to second visual measurement data, wherein the second visual measurement data is pose data of the marker in a gait reference coordinate system of the robot; and
performing a staged gait planning on a process of the robot to climb the staircase based on the second visual measurement data.

### Claim 2 (depends on 1)

The method of claim 1, wherein the step of performing the staged gait planning comprises:
in a flat ground stage, setting first modify pose data of the robot;
calculating a first desired pose data based on the second visual measurement data and the first modify pose data; and
performing the gait planning for the flat ground stage based on the first desire pose data.

### Claim 3 (depends on 2)

The method of claim 2, wherein the step of performing the gait planning for the flat ground stage based on the first desire pose data comprises:
performing the gait planning for the flat ground stage in each of a forward direction, a lateral direction, a height direction, and a yaw direction, based on the formula of:
gvel
=
{
g
⁢
⁢
0
,
g
des
>
threshold
⁢
⁢
1
-
g
⁢
⁢
0
,
g
des
<
-
threshold
⁢
⁢
1
k
*
g
des
,
threshold
⁢
⁢
⁢
2
≤
abs
⁡
(
g
des
)
≤
threshold
⁢
⁢
1
0
,
abs
⁡
(
g
des
)
<
threshold
⁢
⁢
2
;
wherein, g0 is a preset reference movement amount in the target directions, gdes is a component of the first desire pose data in the target directions, k is a preset adjustment coefficient, threshold1 is a preset first threshold, threshold2 is a preset second threshold and threshold1>threshold2, abs is an absolute value function, and gvel is the movement amount in the target directions.

### Claim 4 (depends on 1)

The method of claim 1, wherein the step of performing the staged gait planning comprises:
setting second modify pose data of the robot in a stair climbing stage;
calculating a second desired pose databased on the second visual measurement data and the second modify pose data; and
performing the gait planning for the stair climbing stage in each of a forward direction, a lateral direction, a height direction, and a yaw direction based on the second desired pose data.

### Claim 5 (depends on 4)

The method of claim 4, wherein the step of performing the gait planning for the stair climbing stage comprises:
performing the gait planning for the stair climbing stage in each of the lateral direction and the yaw direction based on the formula of:
gvel
=
{
g
⁢
⁢
0
,
g
des
>
threshold
⁢
⁢
1
-
g
⁢
⁢
0
,
g
des
<
-
threshold
⁢
⁢
1
k
*
g
des
,
threshold
⁢
⁢
⁢
2
≤
abs
⁡
(
g
des
)
≤
threshold
⁢
⁢
1
0
,
abs
⁡
(
g
des
)
<
threshold
⁢
⁢
2
.

### Claim 6 (depends on 4)

The method of claim 4, further comprising:
dividing a process of the robot climbing each step of the staircase into preset gait stages;
determining an initial state of a swinging leg of the robot at an initial moment of each gait stage and a desired state of the swinging leg at an end moment of each gait stage, wherein both the initial state and the desired state comprise positions in the forward direction and the height direction; and
performing a curve fitting based on the initial state and the desired state to perform the gait planning for the stair climbing stage in the forward direction and the height direction.

### Claim 7 (depends on 6)

The method of claim 6, wherein the step of determining the initial state of the swinging leg of the robot at the initial moment of each gait stage and the desired state of the swinging leg at the end moment of each gait stage comprises:
in a preset first gait stage, setting an initial position in the forward direction to 0, setting an initial position in the height direction to 0, setting an end position in the forward direction to x_offset, and setting an end position in the height direction to a preset value greater than z_dis_old; wherein x_offset is the distance from a toe of a support leg of the robot to a front edge of a next step in the forward direction, and z_dis_old is a component of the last second visual measurement data in the height direction;
in a preset second stage, setting the initial position in the forward direction to x_offset, setting the initial position in the height direction to the end position of the preset first stage, setting the end position in the forward direction to the sum of x_dis_old and x_offset, and setting the end position in the height direction to a value greater the sum of z_dis_old and z_dis_new; wherein x_dis_old is a component of the last second visual measurement data in the forward direction, and z_dis_new is a component of the current second visual measurement data in the height direction; and
in a preset third stage, setting the initial position in the forward direction to the end position of the preset second stage, setting the initial position in the height direction to the end position of the preset second stage, setting the end position in the forward direction to the sum of x_dis_old and x_dis_new, and setting the end position in the height direction to the sum of z_dis_old and z_dis_new; wherein x_dis_new is a component of the current second visual measurement data in the forward direction.

### Claim 8 (depends on 1)

The method of claim 1, Wherein the step of converting the first visual measurement data to second visual measurement data comprises:
converting the first visual measurement data based on the formulas of:
x_dis=x_dis_from_vision−x_cam2foot;
z_dis=z_cam2foot−z_dis_from_vision;
y_dis=y_dis_from_vision−y_cam2foot; and
yaw_dis=yaw_dis_from_vision;
wherein, x_dis_from_vision is a component of the first visual measurement data in a forward direction, y_dis_from_vision is a component of the first visual measurement data in a lateral direction, and z_dis_from_vision is a component of the first visual measurement data in a height direction, yaw_dis_from_vision is a component of the first visual measurement data in a yaw direction, x_dis is a component of the second visual measurement data in the forward direction, and y_dis is a component of the second visual measurement data in the lateral direction, z_dis is a component of the second visual measurement data in the height direction, yaw_dis is a component of the second visual measurement data in the yaw direction, x_cam2foot is the distance from an installation position of the visual sensor to a toe position of a supporting leg in the forward direction, y_cam2foot is the offset of a position of a center of mass of the robot, and z_cam2foot is the distance from the installation position of the visual sensor to the toe position of the supporting leg in the height direction.

### Claim 9 (depends on 2)

The method of claim 2, wherein the step of setting the first modify pose data of the robot in the flat ground stage comprises:
setting the first modify pose data of the robot in the flat ground stage based on the formula of:
mod=x_set,0,z_set,0)T;
wherein, x_set is a distance between a toe of a support leg of the robot and the marker in the forward direction, z_set is a distance between the toe of the support leg and the marker in the height direction, position data in the lateral direction and the yaw direction are 0.

### Claim 10 (depends on 4)

The method of claim 4, wherein the step of setting the second modify pose data of the robot in the stair climbing stage comprises:
setting the second modify pose data of the robot in the stair climbing stage based on the formula of:
mod=(x_set,0,z_set,0)T;
wherein, x_set is a distance between a toe of a support leg of the robot and the marker of a next step in the forward direction, z_set is a distance between the toe of the support leg and the marker of the next step in the height direction, position data in the lateral direction and the yaw direction are 0.

### Claim 11

A robot, comprising:
a memory;
a processor; and
one or more computer programs stored in the memory and executable on the processor, wherein the one or more computer programs comprise:
instructions for obtaining first visual measurement data through a visual sensor of the robot, wherein the first visual measurement data is pose data of a preset marker on a step of a staircase in a body coordinate system of the robot;
instructions for converting the first visual measurement data to second visual measurement data, Wherein the second visual measurement data is pose data of the marker in a gait reference coordinate system of the robot; and
instructions for performing a staged gait planning on a process of the robot to climb the staircase based on the second visual measurement data.

### Claim 12 (depends on 11)

The robot of claim 11, wherein the instructions for performing the staged gait planning comprise:
instructions for in a flat ground stage, setting first modify pose data of the robot;
instructions for calculating a first desired pose data based on the second visual measurement data and the first modify pose data; and
instructions for performing the gait planning for the flat ground stage based on the first desire pose data.

### Claim 13 (depends on 12)

The robot of claim 12, wherein the instructions for performing the gait planning for the flat ground stage based on the first desire pose data comprise:
instructions for performing the gait planning for the flat ground stage in each of a forward direction, a lateral direction, a height direction, and a yaw direction, based on the formula of:
gvel
=
{
g
⁢
⁢
0
,
g
des
>
threshold
⁢
⁢
1
-
g
⁢
⁢
0
,
g
des
<
-
threshold
⁢
⁢
1
k
*
g
des
,
threshold
⁢
⁢
⁢
2
≤
abs
⁡
(
g
des
)
≤
threshold
⁢
⁢
1
0
,
abs
⁡
(
g
des
)
<
threshold
⁢
⁢
2
;
wherein, g0 is a preset reference movement amount in the target directions, gdes is a component of the first desire pose data in the target directions, k is a preset adjustment coefficient, threshold1 is a preset first threshold, threshold2 is a preset second threshold and threshold1>threshold2, abs is an absolute value function, and gvel is the movement amount in the target directions.

### Claim 14 (depends on 11)

The robot of claim 11, wherein the instructions for performing the staged gait planning comprise:
instructions for setting second modify pose data of the robot in a stair climbing stage;
instructions for calculating a second desired pose data based on the second visual measurement data and the second modify pose data; and
instructions for performing the gait planning for the stair climbing stage in each of a forward direction, a lateral direction, a height direction, and a yaw direction based on the second desired pose data.

### Claim 15 (depends on 14)

The robot of claim 14, wherein the instructions for performing the gait planning for the stair climbing stage comprises:
instructions for performing the gait planning for the stair climbing stage in each of the lateral direction and the yaw direction based on the formula of:
gvel
=
{
g
⁢
⁢
0
,
g
des
>
threshold
⁢
⁢
1
-
g
⁢
⁢
0
,
g
des
<
-
threshold
⁢
⁢
1
k
*
g
des
,
threshold
⁢
⁢
⁢
2
≤
abs
⁡
(
g
des
)
≤
threshold
⁢
⁢
1
0
,
abs
⁡
(
g
des
)
<
threshold
⁢
⁢
2
.

### Claim 16 (depends on 14)

The robot of claim 14, wherein the one or more computer programs further comprise:
instructions for dividing a process of the robot climbing each step of the staircase into preset gait stages;
instructions for determining an initial state of a swinging leg of the robot at an initial moment of each gait stage and a desired state of the swinging leg at an end moment of each gait stage, wherein both the initial state and the desired state comprise positions in the forward direction and the height direction; and
instructions for performing a curve fitting based on the initial state and the desired state to perform the gait planning for the stair climbing stage in the forward direction and the height direction.

### Claim 17 (depends on 16)

The robot of claim 16, wherein the instructions for determining the initial state of the swinging leg of the robot at the initial moment of each gait stage and the desired state of the swinging leg at the end moment of each gait stage comprise:
instructions for in a preset first gait stage, setting an initial position in the forward direction to 0, setting an initial position in the height direction to 0, setting an end position in the forward direction to x_offset, and setting an end position in the height direction to a preset value greater than z_dis_old; wherein x_offset is the distance from a toe of a support leg of the robot to a front edge of a next step in the forward direction, and z_dis_old is a component of the last second visual measurement data in the height direction;
instructions for in a preset second stage, setting the initial position in the forward direction to x_offset, setting the initial position in the height direction to the end position of the preset first stage, setting the end position in the forward direction to the sum of x_dis_old and x_offset, and setting the end position in the height direction to a value greater the sum of z_dis_old and z_dis_new; wherein x_dis_old is a component of the last second visual measurement data in the forward direction, and z_dis_new is a component of the current second visual measurement data in the height direction; and
instructions for in a preset third stage, setting the initial position in the forward direction to the end position of the preset second stage, setting the initial position in the height direction to the end position of the preset second stage, setting the end position in the forward direction to the sum of x_dis_old and x_dis_new, and setting the end position in the height direction to the sum of z_dis_old and z_dis_new; wherein x_dis_new is a component of the current second visual measurement data in the forward direction.

### Claim 18 (depends on 11)

The robot of claim 11, wherein the instructions for converting the first visual measurement data to second visual measurement data comprise:
instructions for converting the first visual measurement data based on the formulas of:
x_dis=x_dis_from_vision−x_cam2foot;
z_dis=z_cam2foot−z_dis_from_vision;
y_dis=y_dis_from_vision−y_cam2foot; and
yaw_dis=yaw_dis_from_vision;
wherein, x_dis_from_vision is a component of the first visual measurement data in a forward direction, y_dis_from_vision is a component of the first visual measurement data in a lateral direction, and z_dis_from_vision is a component of the first visual measurement data in a height direction, yaw_dis_from_vision is a component of the first visual measurement data in a yaw direction, x_dis is a component of the second visual measurement data in the forward direction, and y_dis is a component of the second visual measurement data in the lateral direction, z_dis is a component of the second visual measurement data in the height direction, yaw_dis is a component of the second visual measurement data in the yaw direction, x_cam2foot is the distance from an installation position of the visual sensor to a toe position of a supporting leg in the forward direction, y_cam2foot is the offset of a position of a center of mass of the robot, and z_cam2foot is the distance from the installation position of the visual sensor to the toe position of the supporting leg in the height direction.

### Claim 19 (depends on 12)

The robot of claim 12, wherein the instructions for setting the first modify pose data of the robot in the flat ground stage comprise:
instructions for setting the first modify pose data of the robot in the flat ground stage based on the formula of:
mod=(x_set,0,z_set,0)T;
wherein, x_set is a distance between a toe of a support leg of the robot and the marker in the forward direction, z_set is a distance between the toe of the support leg and the marker in the height direction, position data in the lateral direction and the yaw direction are 0.

### Claim 20 (depends on 14)

The robot of claim 14, wherein the instructions for setting the second modify pose data of the robot in the stair climbing stage comprise:
instructions for setting the second modify pose data of the robot in the stair climbing stage based on the formula of:
mod=(x_set,0,z_set,0)T;
wherein, x_set is a distance between a toe of a support leg of the robot and the marker of a next step in the forward direction, z_set is a distance between the toe of the support leg and the marker of the next step in the height direction, position data in the lateral direction and the yaw direction are 0.
