# US 2017/0361466 A1 · Vision-based robot control system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0361466 A1
- Document ID: US20170361466A1
- Status: Published application
- Kind: A1
- Application number: 15/185,241
- Filed: 2016-06-17
- Published: 2017-12-21
- Earliest claimed: 2016-06-17
- Assignees: Intel Corporation
- Inventors: Glen J. Anderson
- Web page: https://patentgrep.com/patent/US20170361466A1

## CPC

- B25J 9/1697: {Vision controlled systems}
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})

## Abstract

Various systems and methods for providing a vision-based robot control system are provided herein. A vision-based robot control system comprises a camera system interface to receive image data from a camera system; a trigger detection unit to determine a triggering action from the image data; and a transceiver to initiate a robot operation associated with the triggering action.

## Claims

### Claim 1

A vision-based robot control system, the system comprising:
a camera system interface to receive image data from a camera system;
a trigger detection unit to determine a triggering action from the image data; and
a transceiver to initiate a robot operation associated with the triggering action.

### Claim 2 (depends on 1)

The system of claim 1, wherein to receive the image data, the camera system interface is to receive an image of a user performing a gesture; and
wherein to determine the triggering action, the trigger detection unit is to determine the triggering action corresponding to the gesture.

### Claim 3 (depends on 2)

The system of claim 2, wherein the gesture comprises pointing to a location in an environment.

### Claim 4 (depends on 2)

The system of claim 2, wherein the gesture comprises tapping a foot on a location in an environment.

### Claim 5 (depends on 4)

The system of claim 4, wherein the triggering action is the gesture and the robot operation comprises performing extra cleaning at the location in the environment.

### Claim 6 (depends on 1)

The system of claim 1, wherein the image data includes a gesture performed by a user, and wherein the trigger detection unit is to receive a voice command issued by the user and determine the triggering action using the gesture and the voice command

### Claim 7 (depends on 6)

The system of claim 6, wherein the gesture is used to specify a location in the environment, and wherein the voice command is used to specify an action to be taken by the robot.

### Claim 8 (depends on 1)

The system of claim 1, wherein the image data includes a non-visible marking made by a user, the non-visible marking acting as the triggering action.

### Claim 9 (depends on 8)

The system of claim 8, wherein the non-visible marking comprises an infrared-visible ink marking.

### Claim 10 (depends on 8)

The system of claim 8, wherein the non-visible marking is a symbol, and to initiate the robot operation, an operation database interface is used to access a lookup table and search for the symbol to identify a corresponding robot operation.

### Claim 11 (depends on 1)

The system of claim 1, wherein the trigger detection unit is to obtain a geolocation associated with the triggering action; and wherein the transceiver is to initiate the robot operation to be performed at the geolocation.

### Claim 12 (depends on 11)

The system of claim 11, wherein the geolocation is obtained from a device operated by a user performing the triggering action.

### Claim 13 (depends on 11)

The system of claim 11, wherein the geolocation is obtained from a gesture performed by the user, the gesture captured in the image data.

### Claim 14 (depends on 1)

The system of claim 1, wherein to initiate the robot operation, the transceiver is to transmit a command sequence to a robot over a wireless network.

### Claim 15 (depends on 1)

The system of claim 1, wherein the robot operation comprises a cleaning task.

### Claim 16

A method of providing a vision-based robot control system, the method comprising:
receiving, at a processor-based robot control system, image data from a camera system;
determining a triggering action from the image data; and
initiating a robot operation associated with the triggering action.

### Claim 17 (depends on 16)

The method of claim 16, wherein receiving the image data comprises receiving an image of a user performing a gesture; and
wherein determining the triggering action comprises determining the triggering action corresponding to the gesture.

### Claim 18 (depends on 17)

The method of claim 17, wherein the gesture comprises pointing to a location in an environment.

### Claim 19 (depends on 17)

The method of claim 17, wherein the gesture comprises tapping a foot on a location in an environment.

### Claim 20 (depends on 19)

The method of claim 19, wherein the triggering action is the gesture and the robot operation comprises performing extra cleaning at the location in the environment.

### Claim 21

At least one machine-readable medium including instructions, which when executed by a machine, cause the machine to
receive, at a processor-based robot control system, image data from a camera system;
determine a triggering action from the image data; and
initiate a robot operation associated with the triggering action.

### Claim 22 (depends on 21)

The at least one machine-readable medium of claim 21, wherein the instructions to receive the image data comprise instructions to receive an image of a user performing a gesture; and
wherein the instructions to determine the triggering action comprise instructions to determine the triggering action corresponding to the gesture.

### Claim 23 (depends on 22)

The at least one machine-readable medium of claim 22, wherein the gesture comprises pointing to a location in an environment.

### Claim 24 (depends on 22)

The at least one machine-readable medium of claim 22, wherein the gesture comprises tapping a foot on a location in an environment.

### Claim 25 (depends on 24)

The at least one machine-readable medium of claim 24, wherein the triggering action is the gesture and the robot operation comprises performing extra cleaning at the location in the environment.
