# US 2012/0253582 A1 · Semi-Autonomous Mobile Device Driving with Obstacle Avoidance

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2012/0253582 A1
- Document ID: US20120253582A1
- Status: Published application
- Kind: A1
- Application number: 13/076,032
- Filed: 2011-03-30
- Published: 2012-10-04
- Earliest claimed: 2011-03-30
- Assignees: Microsoft Corporation
- Inventors: Georgios Chrysanthakopoulos; Michael J. Wyrzykowski
- Web page: https://patentgrep.com/patent/US20120253582A1

## CPC

- G05D 1/0274: {using mapping information stored in a memory device (navigation using map-matching G01C21/30)}
- G05D 1/024: {in combination with a laser (lasers per se H01S)}
- G05D 1/0242: {using non-visible light signals, e.g. IR or UV signals}
- G05D 1/0255: {using acoustic signals, e.g. ultra-sonic singals (sonar systems designed for anti-collision purposes G01S15/93)}
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES

## Abstract

The subject disclosure is directed towards driving a robot or other mobile device safely through an environment by using a depth camera to obtain depth data, and then using the depth data for collision avoidance. Horizontal profile information may be built from the depth data, such as by collapsing a two-dimensional depth map into one-dimensional horizontal profile information. The horizontal profile information may be further processed by fusing the depth camera-detected obstacle data with any closer obstacle data as detected via infrared-based sensing and/or sonar-based sensing. Driving suggestions from a user or program are overridden as needed to avoid collisions, including by driving the robot towards an open space represented in the horizontal profile information, or stopping/slowing the robot as needed.

## Claims

### Claim 1

In a computing environment, a system comprising, a depth camera configured to capture depth data, a navigation mechanism configured to process the depth data into an obstacle map, and navigation logic configured to drive a mobile device through an environment based upon the obstacle map to avoid colliding with any obstacle represented in the obstacle map.

### Claim 2 (depends on 1)

The system of claim 1 wherein the navigation mechanism is further configured to receive one or more driving suggestions from a user or program, and wherein the navigation logic uses the one or more driving suggestions in conjunction with the obstacle map to drive the mobile device based upon the driving suggestions, or based upon any obstacle represented in the obstacle map, or based upon both the driving suggestions and the presence of any obstacle represented in the obstacle map.

### Claim 3 (depends on 1)

The system of claim 1 wherein the mobile device comprises a robot configured to be driven at a linear speed and an angular speed.

### Claim 4 (depends on 3)

The system of claim 3 wherein the navigation logic evaluates whether the angular speed is sufficient to rotate the robot if rotation is to occur, and if not, the navigation logic configured to modify the angular speed to a value that is sufficient to rotate the robot.

### Claim 5 (depends on 1)

The system of claim 1 wherein the depth data comprises a two-dimensional map, and wherein the navigation mechanism processes the depth data into the obstacle map comprising a one-dimensional horizontal vector.

### Claim 6 (depends on 1)

The system of claim 1 further comprising one or more infrared sensors, and wherein the navigation mechanism processes the depth data into the obstacle map based on a combination of the depth data and data from at least one of the one or more infrared sensors.

### Claim 7 (depends on 1)

The system of claim 1 further comprising a sonar mechanism, and wherein the navigation mechanism processes the depth data into the obstacle map based on a combination of the depth data and data from the sonar mechanism.

### Claim 8 (depends on 1)

The system of claim 1 further comprising one or more range detecting sensors, wherein the navigation mechanism processes the depth data into the obstacle map based on a combination of the depth data and data from at least one of the one or more range detecting sensors.

### Claim 9 (depends on 1)

The system of claim 1 further comprising a sensor that detects when an obstacle is behind the mobile device.

### Claim 10 (depends on 1)

The system of claim 1 wherein the navigation logic is configured to drive the mobile device to avoid colliding with any obstacle represented in the obstacle map by driving the mobile device towards an open space represented in the obstacle map.

### Claim 11 (depends on 1)

The system of claim 1 wherein the navigation logic is configured to drive the mobile device to avoid colliding with any obstacle represented in the obstacle map by driving the mobile device towards a midpoint of an open space represented in the obstacle map.

### Claim 12

In a computing environment, a method performed at least in part on at least one processor, comprising:
capturing depth data via a depth camera, the depth data representative of obstacles in an environment to be navigated by a mobile device;
receiving one or more suggested driving commands from a user or program;
using the one or more suggested driving commands and profile information corresponding to the depth data to navigate the mobile device through the environment, including by driving the mobile device according to the suggested driving commands until the profile information indicates an anticipated collision with an obstacle; and
taking action to avoid an actual collision between the mobile device and the obstacle when the profile information indicates an anticipated collision.

### Claim 13 (depends on 12)

The method of claim 12 further comprising, processing the depth data into the profile information, including combining the profile information based on the depth data with data from at least one other sensor.

### Claim 14 (depends on 12)

The method of claim 12 wherein taking the action comprises slowing the mobile device linear speed or angular speed, or both linear speed and angular speed, driving the mobile device towards an open space represented in the profile information, or stopping the mobile device with respect to linear speed or angular speed, or both linear speed and angular speed.

### Claim 15 (depends on 12)

The method of claim 12 wherein taking the action comprises stopping the mobile device with respect to linear speed, and slowing the mobile device angular speed.

### Claim 16

One or more computer-readable media having computer-executable instructions, which when executed perform steps, comprising:
capturing depth data via a depth camera, the depth data representative of obstacles in an environment to be navigated by a mobile device;
processing the depth data into profile information;
modifying the profile information based upon sonar-based obstacle sensing; and
using the profile information to navigate the mobile device through the environment, including by taking action to avoid an actual collision when the profile information indicates an anticipated collision between the mobile device and the obstacle.

### Claim 17 (depends on 16)

The one or more computer-readable media of claim 16 having further computer-executable instructions comprising, modifying the profile information based upon infrared-based obstacle sensing.

### Claim 18 (depends on 16)

The one or more computer-readable media of claim 16 having further computer-executable instructions comprising receiving one or more suggested driving commands from a user or program, and driving the mobile device according to the suggested driving commands until the profile information indicates an anticipated collision with an obstacle.

### Claim 19 (depends on 16)

The one or more computer-readable media of claim 16 wherein the depth data comprises a two-dimensional map of columns and rows, and wherein processing the depth data into the profile information comprises selecting a single value from each column, including selecting from each column a value that represents a minimum distance to an obstacle with which the mobile device is able to collide.

### Claim 20 (depends on 16)

The one or more computer-readable media of claim 16 wherein taking the action comprises slowing the mobile device linear speed, angular speed, or both linear speed and angular speed, driving the mobile device towards an open space represented in the profile information, or stopping the mobile device with respect to linear speed angular speed, or both linear speed and angular speed.
