# US 9,987,748 B2 · Networked robotic manipulators

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 9,987,748 B2
- Document ID: US9987748B2
- Status: Granted
- Kind: B2
- Application number: 15/632,140
- Filed: 2017-06-23
- Published: 2017-10-19
- Granted: 2018-06-05
- Earliest claimed: 2015-09-21
- Assignees: Amazon Technologies, Inc.
- Applicants: Amazon Technologies, Inc.
- Inventors: Andrew Stubbs; Diane Grieselhuber Mills; John Gregory Longtine; Matthew David Verminski
- Web page: https://patentgrep.com/patent/US9987748B2

## CPC

- B25J 9/1679: {characterised by the tasks executed}
- B25J 9/163: {learning, adaptive, model based, rule based expert control}
- B25J 9/1682: {Dual arm manipulator; Coordination of several manipulators}
- G05B 19/418: Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS] or computer integrated manufacturing [CIM]
- G06Q 10/087: Inventory or stock management, e.g. order filling, procurement or balancing against orders
- G06Q 10/0877: {by inventory control or reporting using inventory tracking or counting}
- G05B 2219/39106: Conveyor, pick up article, object from conveyor, bring to test unit, place it
- G05B 2219/39107: Pick up article, object, measure, test it during motion path, place it
- G05B 2219/40053: Pick 3-D object from pile of objects
- G06Q 10/08726: {for replenishment processing, procedures, or recommendations using forecasting or optimisation}
- Y02P 90/02: Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
- Y10S 901/02: Arm motion controller
- Y10S 901/46: Sensing device
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- Y02P: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y10S: TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS

## Abstract

Robotic manipulators may be used to manipulate objects. Manipulation data about manipulations performed on objects may be generated and accessed. This data may be analyzed to generate a profile indicating how an object may be manipulated. A portion of the profile may be transmitted to a particular robotic manipulator. For example, the portion may be based on a manipulation capability of the robotic manipulator. In turn, the robotic manipulator may use the portion of the profile to manipulate the object.

## Claims

### Claim 1

An inventory system, comprising:
one or more processors; and
one or more computer-readable media comprising computer-readable instructions that, upon execution by the one or more processors, cause the inventory system to at least:
receive, from a first robotic manipulator, first data about performing an action by the first robotic manipulator, the action associated with manipulating an inventory item;
generate a manipulation profile based at least in part on the first data, the manipulation profile comprising instructions about manipulating the inventory item by performing the action;
transmit a portion of the manipulation profile to a second robotic manipulator based at least in part on the action being supported by the second robotic manipulator, the portion comprising the instructions;
receive, from the second robotic manipulator, second data about performing the action by the second robotic manipulator, the second data comprising an indication of a failure of the second robotic manipulator to perform the action according to the instructions; and
update the instructions in the manipulation profile based at least in part on the second data, the update comprising changing the instructions based at least in part on the failure.

### Claim 2 (depends on 1)

The inventory system of claim 1, wherein the first robotic manipulator is configured to perform a different action that is associated with manipulating the inventory item and that is unsupported by the second robotic manipulator, and wherein the execution of the computer-readable instructions further cause the inventory system to at least:
receive, from the first robotic manipulator, additional data about performing the different action by the first robotic manipulator; and
add, to the manipulation profile, additional instructions about performing the different action based at least in part on the additional data.

### Claim 3 (depends on 2)

The inventory system of claim 2, wherein the portion of the manipulation profile transmitted to the second robotic manipulator excludes the additional instructions based at least in part on the different action being unsupported by the second robotic manipulator.

### Claim 4 (depends on 1)

The inventory system of claim 1, wherein the second robotic manipulator is configured to perform a different action that is associated with manipulating the inventory item and that is unsupported by the first robotic manipulator, and wherein the execution of the computer-readable instructions further cause the inventory system to at least:
receive, from the second robotic manipulator, additional data about performing the different action by the second robotic manipulator; and
add, to the manipulation profile, additional instructions about performing the different action based at least in part on the additional data.

### Claim 5 (depends on 4)

The inventory system of claim 4, wherein the execution of the computer-readable instructions further cause the inventory system to at least:
transmit the instructions and the additional instructions of the manipulation profile to a third robotic manipulator based at least in part on a determination that the third robotic manipulator supports the action and the different action;
receive, from the third robotic manipulator, third data about performing the action and the different action by the third robotic manipulator; and
further update the instructions in the manipulation profile based at least in part on the third data.

### Claim 6 (depends on 4)

The inventory system of claim 4, wherein the execution of the computer-readable instructions further cause the inventory system to at least:
access a profile of the second robotic manipulator, the profile indicating that the second robotic manipulator supports the action and the different action; and
select the portion of the manipulation profile based at least in part on the profile of the second robotic manipulator, the portion comprising the instructions and the additional instructions.

### Claim 7

A computer-implemented method, comprising:
accessing, by a computer system, first manipulation data of a first robotic manipulator about a manipulation of an object type;
generating, by the computer system, instructions associated with manipulating the object type based at least in part on the first manipulation data;
transmitting, by the computer system, the instructions to a second robotic manipulator that supports the manipulation;
receiving, by the computer system from the second robotic manipulator, second manipulation data of the second robotic manipulator about the manipulation, the second manipulation data generated based at least in part on the instructions, the second manipulation data comprising an indication of a failure of the second robotic manipulator to perform the manipulation according to the instructions; and
updating, by the computer system, the instructions based at least in part on the second manipulation data of the second robotic manipulator, the updating comprising changing the instructions based at least in part on the failure.

### Claim 8 (depends on 7)

The computer-implemented method of claim 7, wherein updating the instructions comprises:
sending a request to the first robotic manipulator to confirm the failure of the manipulation at the first robotic manipulator; and
changing the instructions based at least in part on a confirmation of the failure from the first robotic manipulator.

### Claim 9 (depends on 7)

The computer-implemented method of claim 7, wherein generating the instructions comprises generating a manipulation profile, and wherein transmitting the instructions comprises:
selecting a portion of the manipulation profile based at least in part on a determination that the second robotic manipulator supports the manipulation, the portion comprising the instructions; and
transmitting the portion of the manipulation profile to the second robotic manipulator.

### Claim 10 (depends on 9)

The computer-implemented method of claim 9, wherein selecting the portion of the manipulation profile comprises:
determining, based at least in part on profiles of the first robotic manipulator and second robotic manipulator, that a different manipulation of the object type is supported by the first robotic manipulator and is unsupported by the second robotic manipulator; and
excluding, from the portion of the manipulation profile, different instructions about the different manipulation, the different instructions being available from the manipulation profile based at least in part on different manipulation data of the first robotic manipulator about the different manipulation.

### Claim 11 (depends on 7)

The computer-implemented method of claim 7, further comprising:
requesting, by the computer system from the second robotic manipulator, an indication of whether an attribute associated with the object type has changed, the attribute being common to objects of the object type and impacting the manipulation of the object type;
receiving, by the computer system from the second robotic manipulator based at least in part on the requesting, information that a first set of the objects comprises the attribute unchanged and that a second set of the objects comprises the attribute changed; and
generating, by the computer system, second instructions associated with manipulating the second set of the objects based at least in part on a change to the attribute.

### Claim 12

One or more non-transitory computer-readable storage media comprising computer-readable instructions that, upon execution by one or more processors, cause a system to perform operations comprising:
accessing first manipulation data of a first robotic manipulator about a manipulation of an object type, the manipulation of the object type associated with an attribute common to objects of the object type;
generating a profile associated with manipulating the object type based at least in part on the first manipulation data, a portion of the profile comprising instructions about the manipulation;
transmitting the portion of the profile to a second robotic manipulator based at least in part on the manipulation being supported by the second robotic manipulator;
receiving, from the second robotic manipulator, second manipulation data of the second robotic manipulator about the manipulation, the second manipulation data generated based at least in part on the portion of the profile;
updating the portion the profile based at least in part on the second manipulation data of the second robotic manipulator;
receiving, from the second robotic manipulator, information that a first set of the objects comprises the attribute unchanged and that a second set of the objects comprises the attribute changed; and
generating a second profile associated with manipulating the second set of the objects based at least in part on a change to the attribute.

### Claim 13 (depends on 12)

The one or more non-transitory computer-readable storage media of claim 12, wherein generating the profile comprises adding different instructions to the profile about a different manipulation of the object type, wherein the different manipulation is supported by the first robotic manipulator and is unsupported by the second robotic manipulator, and wherein the portion transmitted to the second robotic manipulator excludes the different instructions based at least in part on the different manipulation being unsupported by the second robotic manipulator.

### Claim 14 (depends on 13)

The one or more non-transitory computer-readable storage media of claim 13, further comprising:
receiving, from the second robotic manipulator, additional manipulation data about an additional manipulation of the object type, wherein the additional manipulation is supported by the second robotic manipulator; and
adding, to the profile, additional instructions about the additional manipulation based at least in part on the additional manipulation data.

### Claim 15 (depends on 12)

The one or more non-transitory computer-readable storage media of claim 12, wherein receiving the second manipulation data comprises receiving an indication of a failure of the second robotic manipulator to perform the manipulation according to the portion of the profile, and wherein updating the instructions comprises changing the instructions based at least in part on the failure.

### Claim 16 (depends on 15)

The one or more non-transitory computer-readable storage media of claim 15, wherein updating the instructions comprises:
sending a request to the first robotic manipulator to confirm the failure of the manipulation at the first robotic manipulator; and
changing the instructions in the profile based at least in part on a confirmation of the failure from the first robotic manipulator.

### Claim 17 (depends on 12)

The one or more non-transitory computer-readable storage media of claim 12, wherein transmitting the portion of the profile comprises:
selecting the instructions from the profile based at least in part on a determination that the second robotic manipulator supports the manipulation of the object type and based at least in part on an association between the instructions and the manipulation.

### Claim 18 (depends on 17)

The one or more non-transitory computer-readable storage media of claim 17, wherein the determination that the second robotic manipulator supports the manipulation is based at least in part on a component of the second robotic manipulator, wherein the component is configured to manipulate objects of the object type and comprises at least one of a robotic arm or an end effector.