# US 2003/0216834 A1 · Method and system for remote control of mobile robot

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2003/0216834 A1
- Document ID: US20030216834A1
- Status: Published application
- Kind: A1
- Application number: 10/339,168
- Filed: 2003-01-09
- Published: 2003-11-20
- Earliest claimed: 2000-05-01
- Assignees: iRobot Corporation
- Inventors: James R. Allard
- Web page: https://patentgrep.com/patent/US20030216834A1

## CPC

- G05D 1/0038: {by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation (images analyzed by a computer and used for automatic navigation G05D1/0246)}
- B25J 9/1689: {Teleoperation}
- G05D 1/0044: {by providing the operator with a computer generated representation of the environment of the vehicle, e.g. virtual reality, maps (maps used for automatic navigation G05D1/0274; flight directors G01C23/005)}
- G05D 1/0246: {using a video camera in combination with image processing means}
- G05D 1/0282: {generated in a local control room}
- G05B 2219/35506: Camera images overlayed with graphics, model
- G05B 2219/40161: Visual display of machining, operation, remote viewing
- G05B 2219/40169: Display of actual situation at the remote site
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)

## Abstract

A system for tele-operating a robot in an environment includes a user interface for controlling the tele-operation of the robot, an imaging device associated with the robot for providing image information representative of the environment around the robot, means for transmitting the image information to the user interface, means for converting the image information to a user-perceptible image at the user interface, means for designating one or more waypoints located anywhere in the user-perceptible image towards which the robot will move, the waypoint in the user-perceptible image towards which the robot will first move being designated as the active waypoint using an icon, means for automatically converting the location of the active waypoint in the user-perceptible image into a target location having x, y, and z coordinates in the environment of the robot, means for providing real-time instructions to the robot from the user interface to move the robot from the robot's current location in the environment to the x, y, and z coordinates of the target location in the environment, and means for moving the icon representing the active waypoint in the user-perceptible image to a new location in the user-perceptible image while the robot is executing the real-time instruction, wherein the location-converting means automatically converts the new location of the icon representing the active waypoint into a new target location having x, y, and z coordinates in the environment of the robot towards which the robot will move.

## Claims

### Claim 1

A system for tele-operating a robot in an environment, comprising:
a user interface for controlling the tele-operation of the robot;
an imaging device associated with the robot for providing image information representative of the environment around the robot;
means for transmitting the image information to the user interface;
means for converting the image information to a user-perceptible image at the user interface;
means for designating one or more waypoints located anywhere in the user-perceptible image towards which the robot will move, the waypoint in the user-perceptible image towards which the robot will first move being designated as the active waypoint using an icon;
means for automatically converting the location of the active waypoint in the user-perceptible image into a target location having x, y, and z coordinates in the environment of the robot;
means for providing real-time instructions to the robot from the user interface to move the robot from the robot's current location in the environment to the x, y, and z coordinates of the target location in the environment; and
means for moving the icon representing the active waypoint in the user-perceptible image to a new location in the user-perceptible image while the robot is executing the real-time instructions;
wherein the location-converting means automatically converts the new location of the icon representing the active waypoint into a new target location having x, y, and z coordinates in the environment of the robot towards which the robot will move.

### Claim 2 (depends on 1)

The robot tele-operating system of claim 1 wherein the active waypoint moving means comprises:
means for selecting and dragging the icon representing the active waypoint to the new location in the user-perceptible image; and
means for dropping the icon at the new location in the user-perceptible image.

### Claim 3 (depends on 2)

The robot tele-operating system of claim 2 wherein the active waypoint moving means comprises a mouse having a click button configured to act as said selecting and dragging means and said dropping means wherein depressing and holding of the click button permits a user to select and drag the icon representing the active waypoint to the new location in the user-perceptible image and releasing of the click button drops the icon representing the active waypoint at the new location in the user-perceptible image.

### Claim 4 (depends on 2)

The robot tele-operating system of claim 2 wherein the active waypoint moving means comprises:
means for moving the icon representing the active waypoint a predetermined amount in the user-perceptible image to the new location wherein the predetermined amount of movement of the icon is with respect to a directional vector defined between the robot's current location and the location of the active waypoint.

### Claim 5 (depends on 4)

The robot tele-operating system of claim 4 wherein the icon-moving means is operative to move the icon representing the active waypoint the predetermined amount along the directional vector away from or towards the robot.

### Claim 6 (depends on 4)

The robot tele-operating system of claim 4 wherein the icon-moving means is operative to move the icon representing the active waypoint the predetermined amount orthogonal to the directional vector.

### Claim 7 (depends on 4)

The robot tele-operating system of claim 4 wherein the icon-moving means is operative to move the icon representing the active waypoint the predetermined amount along the directional vector away from or towards the robot and to move the icon representing the active waypoint the predetermined amount orthogonal to the directional vector.

### Claim 8 (depends on 2)

The robot tele-operating system of claim 2 wherein the active waypoint moving means comprises:
means for rotating the icon representing the active waypoint a predetermined angle in the user-perceptible image to the new location wherein the predetermined angle of rotation of the icon is with respect to a directional vector defined between the robot's current location and the location of the active waypoint.

### Claim 9 (depends on 1)

The robot tele-operating system of claim 1 wherein the real-time instructions prohibit movement of the robot in the environment until the active waypoint has completed movement to the new location in the user-perceptible image.

### Claim 10 (depends on 1)

The robot tele-operating system of claim 1 wherein the real-time instructions allow movement of the robot in the environment while the active waypoint is being moved to the new location in the user-perceptible image.
