# US 12,216,477 B2 · Robot and controlling method thereof

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,216,477 B2
- Document ID: US12216477B2
- Status: Granted
- Kind: B2
- Application number: 17/843,394
- Filed: 2022-06-17
- Published: 2023-03-09
- Granted: 2025-02-04
- Earliest claimed: 2021-09-07
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Dohoon Kim; Aron Baik; Chanho Yoon; Eunsoll Chang
- Web page: https://patentgrep.com/patent/US12216477B2

## CPC

- G05D 1/0238: {using obstacle or wall sensors (G05D1/0246 and G05D1/0289 take precedence; lidar systems designed for anti-collision purposes G01S17/93)}
- B25J 9/16: Program controls (total factory control, i.e. centrally controlling a plurality of machines, G05B19/418)
- B25J 13/08: by means of sensing devices, e.g. viewing or touching devices
- B25J 19/02: Sensing devices
- G05D 1/02: Control of position or course in two dimensions
- G05D 1/0214: {in accordance with safety or protection criteria, e.g. avoiding hazardous areas (monitoring the location of vehicles within a certain area, e.g. forbidden or allowed areas, in traffic control systems for road vehicles G08G1/13)}
- G05D 1/024: {in combination with a laser (lasers per se H01S)}
- G05D 1/0274: {using mapping information stored in a memory device (navigation using map-matching G01C21/30)}
- G05D 1/246: using environment maps, e.g. simultaneous localisation and mapping [SLAM]
- G05D 1/617: Safety or protection, e.g. defining protection zones around obstacles or avoiding hazards (arrangements for controlling the position or course of two or more vehicles for avoiding collisions therebetween G05D1/693; arrangements for reacting to or preventing system or operator failure G05D1/80)
- G05D 1/628: following the obstacle profile, e.g. a wall or undulated terrain
- G06T 7/593: from stereo images
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A robot is provided. The robot includes a driving part, a three dimensional (3D) depth sensor, a memory storing instructions, and a processor connected to the driving part, the 3D depth sensor, and the memory. The processor is configured to execute the instructions to acquire a depth image of a driving surface photographed by the 3D depth sensor while the robot is driving in a space, acquire, based on the acquired depth image, location information of a boundary area where tilt information of the driving surface is changed, acquire type information corresponding to an outside area of the boundary area based on the acquired location information of the boundary area and the changed tilt information, and control the driving part based on the acquired type information.

## Claims

### Claim 1

A robot comprising:
a driving part;
a three dimensional (3D) depth sensor;
a memory storing instructions; and
a processor connected to the driving part, the 3D depth sensor, and the memory,
wherein the processor is configured to execute the instructions to:
acquire a depth image of a driving surface photographed by the 3D depth sensor while the robot is driving in a space,
acquire, based on the acquired depth image, location information of a boundary area by:
acquiring map data comprising a plurality of cells and height information corresponding to the plurality of cells based on the depth image, and
grouping the plurality of cells into a first group having a first value, a second group having a second value different from the first value, and a third group having a third value different from the first value and the second value,
determine changed tilt information of the boundary area based on differences between the first value, the second value and the third value,
acquire type information corresponding to an outside area of the boundary area based on the acquired location information of the boundary area and the changed tilt information, and
control the driving part based on the type information.

### Claim 2 (depends on 1)

The robot of claim 1, wherein the processor is further configured to execute the instructions to:
acquire the location information of the boundary area and the changed tilt information based on the height information of the plurality of cells of the acquired map data.

### Claim 3 (depends on 2)

The robot of claim 2, wherein the processor is further configured to execute the instructions to:
map the type information to the plurality of cells of the outside area,
update the map data,
identify a driving route of the robot based on the updated map data, and
control the driving part based on the identified driving route.

### Claim 4 (depends on 2)

The robot of claim 2, wherein the processor is further configured to execute the instructions to:
group the plurality of cells into at least one group based on the height information of the plurality of cells included in the map data,
identify cells corresponding to the outside area based on the height information of cells belonging to the at least one group, and
identify a type of the outside area based on the height information of the cells identified as corresponding to the outside area.

### Claim 5 (depends on 1)

The robot of claim 1, wherein the processor is further configured to execute the instructions to:
based on the changed tilt information being less than a first threshold value, identify the outside area as a cliff type,
based on the changed tilt information exceeding the first threshold value and being less than a second threshold value, identify the outside area as a slope type, and
based on the changed tilt information being greater than or equal to the second threshold value, identify the outside area as a wall type.

### Claim 6 (depends on 5)

The robot of claim 5, wherein the processor is further configured to execute the instructions to:
based on depth information corresponding to the outside area not being acquired based on the depth image, identify that the changed tilt information is less than the first threshold value.

### Claim 7 (depends on 5)

The robot of claim 5, wherein the processor is further configured to execute the instructions to:
based on the outside area being identified as the cliff type, set an area within a threshold distance from the boundary area as a safe area and control the driving part such that the robot does not drive into the safe area.

### Claim 8 (depends on 5)

The robot of claim 5, wherein the processor is further configured to execute the instructions to:
based on the outside area being identified as the slope type, identify whether the outside area can be climbed by the robot based on tilt information corresponding to the outside area, and
based on the outside area being identified as a climbable area, control the driving part such that a speed of the robot increases based on reaching a threshold distance from the boundary area.

### Claim 9 (depends on 8)

The robot of claim 8, wherein the processor is further configured to execute the instructions to:
based on the outside area being identified as a non-climbable area, control the driving part such that the robot drives based on a route avoiding the outside area.

### Claim 10

A non-transitory computer-readable recording medium storing instructions that, when executed by a processor of a robot, cause the processor to:
acquire a depth image of a driving surface while the robot is driving in a space;
acquire, based on the acquired depth image, location information of a boundary area by:
acquiring map data comprising a plurality of cells and height information corresponding to the plurality of cells based on the depth image, and
grouping the plurality of cells into a first group having a first value, a second group having a second value different from the first value, and a third group having a third value different from the first value and the second value,
determine changed tilt information of the boundary area based on differences between the first value, the second value and the third value,
acquire type information corresponding to an outside area of the boundary area based on the acquired location information of the boundary area and the changed tilt information; and
drive the robot based on the acquired type information.

### Claim 11

A controlling method of a robot, the method comprising:
acquiring a depth image of a driving surface while the robot is driving in a space;
acquiring, based on the acquired depth image, location information of a boundary area by:
acquiring map data comprising a plurality of cells and height information corresponding to the plurality of cells based on the depth image, and
grouping the plurality of cells into a first group having a first value, a second group having a second value different from the first value, and a third group having a third value different from the first value and the second value,
determining changed tilt information of the boundary area based on differences between the first value, the second value and the third value,
acquiring type information corresponding to an outside area of the boundary area based on the acquired location information of the boundary area and the changed tilt information; and
driving the robot based on the acquired type information.

### Claim 12 (depends on 11)

The controlling method of claim 11, wherein the acquiring the location information of the boundary area comprises:
acquiring the location information of the boundary area and the changed tilt information based on the height information of the plurality of cells of the acquired map data.

### Claim 13 (depends on 12)

The controlling method of claim 12, wherein the driving the robot comprises:
mapping the type information to cells included in the outside area,
updating the map data;
identifying a driving route of the robot based on the updated map data; and
driving the robot based on the identified driving route.

### Claim 14 (depends on 12)

The controlling method of claim 12, wherein the acquiring the type information corresponding to the outside area comprises:
grouping the plurality of cells into at least one group based on the height information of the plurality of cells of the map data;
identifying cells corresponding to the outside area based on height information of cells belonging to the at least one group; and
identifying a type of the outside area based on the height information of the cells identified as the outside area.

### Claim 15 (depends on 11)

The controlling method of claim 11, wherein the acquiring the type information corresponding to the outside area comprises:
based on the changed tilt information being less than a first threshold value, identifying the outside area as a cliff type;
based on the changed tilt information exceeding the first threshold value and being smaller than a second threshold value, identifying the outside area as a slope type; and
based on the changed tilt information being greater than or equal to the second threshold value, identifying the outside area as a wall type.

### Claim 16

An apparatus, comprising:
a driving component;
a memory storing instructions; and
a processor configured to execute the instructions to:
acquire a depth image of a driving surface,
acquire, based on the acquired depth image, location information of a boundary area by:
acquiring map data comprising a plurality of cells and height information corresponding to the plurality of cells based on the depth image, and
grouping the plurality of cells into a first group having a first value, a second group having a second value different from the first value, and a third group having a third value different from the first value and the second value,
determine changed tilt information of the boundary area based on differences between the first value, the second value and the third value,
acquire type information of an outside area of the boundary area based on the acquired location information and the changed tilt information, and
control the driving component based on the acquired type information.

### Claim 17 (depends on 16)

The apparatus of claim 16, wherein the processor is further configured to execute the instructions to:
acquire the location information of the boundary area and the changed tilt information based on the height information.

### Claim 18 (depends on 17)

The apparatus of claim 17, wherein the processor is further configured to execute the instructions to:
map the type information to the plurality of cells of the outside area,
update the map data,
identify a driving route of the apparatus based on the updated map data, and
control the driving component based on the identified driving route.

### Claim 19 (depends on 17)

The apparatus of claim 17, wherein the processor is further configured to execute the instructions to:
group the plurality of cells into at least one group based on the height information,
identify cells corresponding to the outside area based on the height information of cells belonging to the at least one group, and
identify a type of the outside area based on the height information of the cells identified as corresponding to the outside area.

### Claim 20 (depends on 16)

The apparatus of claim 16, wherein the processor is further configured to execute the instructions to:
based on the changed tilt information being less than a first threshold value, identify the outside area as a cliff type,
based on the changed tilt information exceeding the first threshold value and being less than a second threshold value, identify the outside area as a slope type, and
based on the changed tilt information being greater than or equal to the second threshold value, identify the outside area as a wall type.