# US 2016/0354927 A1 · Controlling a robot in the presence of a moving object

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0354927 A1
- Document ID: US20160354927A1
- Status: Published application
- Kind: A1
- Application number: 15/224,109
- Filed: 2016-07-29
- Published: 2016-12-08
- Earliest claimed: 2014-02-04
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Harshavardhana Kikkeri; Bryan J. Thibodeau
- Web page: https://patentgrep.com/patent/US20160354927A1

## CPC

- B25J 9/1676: {Avoiding collision or forbidden zones}
- F16P 3/142: {using image capturing devices}
- G06T 1/0014: {Image feed-back for automatic industrial control, e.g. robot with camera (robots B25J19/023)}
- G06T 7/0004: {Industrial image inspection}
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G05B 2219/37567: 3-D vision, stereo vision, with two cameras
- G05B 2219/37571: Camera detecting reflected light from laser
- G05B 2219/39091: Avoid collision with moving obstacles
- G05B 2219/40202: Human robot coexistence
- G05B 2219/40203: Detect position of operator, create non material barrier to protect operator
- G05B 2219/49152: Feedhold, stop motion if machine door is open, if operator in forbidden zone
- G05B 2219/49158: On near collision reduce speed
- G06T 2207/10021: Stereoscopic video; Stereoscopic image sequence
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/20036: Morphological image processing
- G06T 2207/30196: Human being; Person
- H04N 13/204: using stereoscopic image cameras (stereoscopic photography G03B35/00)
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- F16P: SAFETY DEVICES IN GENERAL; {SAFETY DEVICES FOR PRESSES}
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method, system, and one or more computer-readable storage media for controlling a robot in the presence of a moving object are provided herein. The method includes capturing a number of frames from a three-dimensional camera system and analyzing a frame to identify a connected object. The frame is compared to a previous frame to identify a moving connected object (MCO). If an unexpected MCO is in the frame a determination is made if the unexpected MCO is in an actionable region. If so, the robot is instructed to take an action.

## Claims

### Claim 1

A method for controlling a robot in the presence of a moving object, comprising:
capturing a plurality of frames from a three-dimensional camera system;
analyzing a frame to identify a connected object;
comparing the frame to a previous frame to identify a moving connected object (MCO);
determining if an unexpected MCO is in the frame; and, if so, instructing the robot to take an action.
