# US 10,503,143 B1 · Protection system for multi-zone robotic area > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,503,143 B1 - Document ID: US10503143B1 - Status: Granted - Kind: B1 - Application number: 15/946,671 - Filed: 2018-04-05 - Granted: 2019-12-10 - Earliest claimed: 2018-04-05 - Assignees: Amazon Technologies, Inc. - Applicants: Amazon Technologies, Inc. - Inventors: Dennis Polic - Web page: https://patentgrep.com/patent/US10503143B1 ## CPC - G05B 19/406: characterised by monitoring or safety (G05B19/19 takes precedence) - B25J 9/1674: {characterised by safety, monitoring, diagnostic} - B25J 9/1676: {Avoiding collision or forbidden zones} - B25J 9/1697: {Vision controlled systems} - B65G 1/0492: {with cars adapted to travel in storage aisles} - B65G 1/1378: {the orders being assembled on fixed commissioning areas remote from the storage areas} - G05B 19/0425: {Safety, monitoring} - G05B 19/045: using logic state machines, consisting only of a memory or a programmable logic device containing the logic for the controlled machine and in which the state of its outputs is dependent on the state of its inputs or part of its own output states, e.g. binary decision controllers, finite state controllers - G05B 19/048: Monitoring; Safety - G05D 1/244: using passive navigation aids external to the vehicle, e.g. markers, reflectors or magnetic means - G05D 1/617: Safety or protection, e.g. defining protection zones around obstacles or avoiding hazards (arrangements for controlling the position or course of two or more vehicles for avoiding collisions therebetween G05D1/693; arrangements for reacting to or preventing system or operator failure G05D1/80) - G05D 1/667: Delivering or retrieving payloads - B65G 2207/40: Safety features of loads, equipment or persons - G05B 2219/40202: Human robot coexistence - G05D 2105/28: of freight - G05D 2107/70: Industrial sites, e.g. warehouses or factories - G05D 2109/10: Land vehicles - G05D 2111/10: Optical signals - Y10S 901/09: Closed loop, sensor feedback controls arm movement - Y10S 901/31: Gripping jaw - G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G) - B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50}) - B65G: TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS (packaging B65B; handling thin or filamentary materials, e.g. paper sheets or thread, B65H; cranes B66C; portable mobile or lifting or hauling appliances, e.g. hoists, B66D; devices for lifting or lowering goods for loading or unloading purposes, e.g. fork-lift trucks, B66F9/00; emptying bottles, jars, casks, barrels or similar containers, not otherwise provided for, B67C9/00; delivering or transferring liquids B67D; filling or discharging vessels for liquefied, solidified or compressed gases F17C; pipe-line systems for fluids F17D) - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - Y10S: TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS ## Abstract Techniques described herein define a protection system for a multi-zone area where mobile drive units operate autonomously. For example, the protection system may receive sensor data in a particular detection space where the mobile drive units and potentially human operators enter and exit around a robotic arm. The sensor data may identify that an object has entered the detection space around the robotic arm. The sensor data may identify the likelihood of a mobile drive unit entering the area as compared to an operator entering the area. When the object is identified as an operator, any movement of the robotic arm may immediately stop. When the object is identified as the mobile drive unit, protection rules meant to protect the operator may be deactivated so that the mobile drive unit may receive inventory from the robotic arm. ## Claims ### Claim 1 An inventory management system comprising: a mobile drive unit configured to autonomously move in a fulfillment center, the mobile drive unit associated with a height and a width of the mobile drive unit that are stored with a first specification of the mobile drive unit in a data store; a robotic arm configured to pack and stow items moved by the mobile drive unit within the fulfillment center; a plurality of sensors mounted in association with the robotic arm, locations of the plurality of sensors corresponding to the height or the width of the mobile drive unit, and the plurality of sensors being affixed at the locations in association with the robotic arm; and a management module communicatively coupled with the mobile drive unit, the robotic arm, and the plurality of sensors, and configured to: receive, from the plurality of sensors, sensor data in a detection space of the fulfillment center, and the detection space being adjacent to the robotic arm; determine that an object has entered the detection space based at least in part on the sensor data; determine that at least one of the plurality of sensors has changed a state in association with the object entering the detection space; compare the sensor data received from the locations of the plurality of sensors with the specification of the mobile drive unit; determine that a second specification of the object corresponds with the first specification of the mobile drive unit; and when the second specification of the object is identified as the mobile drive unit based at least in part on the first specification and the changed state of the at least one of the plurality of sensors, deactivate protection rules associated with the robotic arm. ### Claim 2 (depends on 1) The inventory management system of claim 1, wherein the protection rules stop movement of the robotic arm or reduce a speed of the robotic arm. ### Claim 3 (depends on 1) The inventory management system of claim 1, the management module further configured to: when the second specification of the object does not correspond with the first specification of the mobile drive unit, determine that the object is a human operator; and stop movement or reduce a speed of the movement of the robotic arm based at least in part on the determination that the object is the human operator. ### Claim 4 (depends on 1) The inventory management system of claim 1, wherein the protection rules comply with ISO 13855. ### Claim 5 A computer system comprising: a mobile drive unit configured to autonomously move in a fulfillment center; a robotic arm configured to pack and stow items moved by the mobile drive unit within the fulfillment center; a plurality of sensors mounted in association with the robotic arm; and a management module communicatively coupled with the mobile drive unit and the plurality of sensors and configured to: receive, from the plurality of sensors, sensor data in a detection space of the fulfillment center, and the detection space being adjacent to the robotic arm; determine that an object has entered the detection space based at least in part on the sensor data; determine that at least one of the plurality of sensors has changed a state in association with the object entering the detection space; and when the object is identified as the mobile drive unit based at least in part on the changed state of the at least one of the plurality of sensors, deactivate protection rules associated with the robotic arm. ### Claim 6 (depends on 5) The computer system of claim 5, wherein the mobile drive unit or an inventory holder carried by the mobile drive unit is associated with a height and a width that are stored with a specification, wherein locations of the plurality of sensors correspond to the height or the width of the mobile drive unit or the inventory holder, and the sensor data identifies a detection of the object received from the locations of the plurality of sensors. ### Claim 7 (depends on 5) The computer system of claim 5, wherein the plurality of sensors further comprise: at least one muting sensor affixed in association with the robotic arm, the muting sensor being configured to temporarily and automatically suspend a safety function of the robotic arm due a non-hazardous portion of movement of the mobile drive unit; and at least one distance sensor affixed in association with the robotic arm, the distance sensor configured to measure a distance in a three-dimensional plane from a mount point adjacent to the robotic arm and toward the detection space. ### Claim 8 (depends on 5) The computer system of claim 5, wherein the plurality of sensors comprise at least a first sensor and a second sensor, and the first sensor is active while the second sensor is inactive, and the first sensor and the second sensor operate asynchronously. ### Claim 9 (depends on 5) The computer system of claim 5, wherein the sensor data comprises first sensor data and second sensor data, and wherein the management module is further configured to: receive the first sensor data in the detection space, the first sensor data identifying a first location of the object; receive the second sensor data in the detection space, the second sensor data identifying a second location of the object; compare the first location of the object and the second location of the object; determine that the object is moving toward the robotic arm; and adjust an operation of the robotic arm based at least in part on the determination. ### Claim 10 (depends on 9) The computer system of claim 9, wherein the management module is further configured to: compare a first width of the object with a second width of the mobile drive unit; and when the first width matches the second width, determine that the mobile drive unit is moving toward the robotic arm. ### Claim 11 (depends on 9) The computer system of claim 9, wherein the management module is further configured to: compare a first height of the object with a second height of a base of an inventory holder being held by the mobile drive unit; and when the first height matches the second height, determine that the mobile drive unit carrying the inventory holder is moving toward the robotic arm. ### Claim 12 (depends on 11) The computer system of claim 11, wherein a specification of the inventory holder is unique in comparison to a human specification corresponding with a human height or a human width. ### Claim 13 A computer-implemented method, comprising: receiving, by a computing system from a plurality of sensors, sensor data in a detection space, and the detection space being adjacent to a robotic arm and the plurality of sensors; determining, by the computing system, that an object has entered the detection space based at least in part on the sensor data; determining, by the computing system, that at least one of the plurality of sensors has changed a state in association with the object entering the detection space; and when the object is identified as a mobile drive unit configured to autonomously move in the detection space based at least in part on the changed state of the at least one of the plurality of sensors, deactivating, by the computing system, protection rules associated with the robotic arm. ### Claim 14 (depends on 13) The computer-implemented method of claim 13, wherein the detection space comprises a plurality of zones that are associated with activation and deactivation of the robotic arm for protection. ### Claim 15 (depends on 13) The computer-implemented method of claim 13, further comprising: detecting an entrance of the mobile drive unit in a first zone of the detection space; detecting a lateral movement of the mobile drive unit in a second zone of the detection space, the lateral movement operating with respect to the robotic arm along a surface of the detection space; detecting an exit of the mobile drive unit in a third zone of the detection space; and in response to detecting the exit, activating protection rules for the first zone, the second zone, and the third zone. ### Claim 16 (depends on 13) The computer-implemented method of claim 13, wherein the determination of the object that entered the detection space is the mobile drive unit that carries an inventory holder in the detection space. ### Claim 17 (depends on 13) The computer-implemented method of claim 13, wherein the plurality of sensors comprise at least a pair of muting sensors and a pair of photoelectric sensors, wherein the pair of muting sensors are associated with deactivating protection rules and the pair of photoelectric sensors are associated with detecting the object in the detection space. ### Claim 18 (depends on 13) The computer-implemented method of claim 13, wherein the plurality of sensors comprise at least a pair of sensors comprising a first sensor of the pair of sensors and a second sensor of the pair of sensors, and wherein the first sensor toggles activation with respect to the second sensor. ### Claim 19 (depends on 18) The computer-implemented method of claim 18, wherein the activation of the second sensor is associated with a delay value after the first sensor becomes inactive. ### Claim 20 (depends on 18) The computer-implemented method of claim 18, wherein the first sensor and the second sensor are directed to point horizontally outward with respect to the robotic arm.