# US 11,440,183 B2 · Hybrid machine learning-based systems and methods for training an object picking robot with real and simulated performance data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,440,183 B2
- Document ID: US11440183B2
- Status: Granted
- Kind: B2
- Application number: 16/366,700
- Filed: 2019-03-27
- Published: 2020-10-01
- Granted: 2022-09-13
- Earliest claimed: 2019-03-27
- Assignees: ABB Schweiz AG
- Applicants: ABB Schweiz AG
- Inventors: Jinmiao Huang; Carlos Martinez; Sangeun Choi; Thomas A. Fuhlbrigge
- Web page: https://patentgrep.com/patent/US11440183B2

## CPC

- B25J 9/163: {learning, adaptive, model based, rule based expert control}
- B25J 9/1612: {characterised by the hand, wrist, grip control}
- B25J 9/1671: {characterised by simulation, either to verify existing program or to create and verify new program, CAD/CAM oriented, graphic oriented programming systems}
- G06F 17/18: for evaluating statistical data {, e.g. average values, frequency distributions, probability functions, regression analysis (forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04)}
- G06N 20/00: Machine learning
- G05B 2219/39473: Autonomous grasping, find, approach, grasp object, sensory motor coordination
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)

## Abstract

For training an object picking robot with real and simulated grasp performance data, grasp locations on an object are assigned based on object physical properties. A simulation experiment for robot grasping is performed using a first set of assigned locations. Based on simulation data from the simulation, a simulated object grasp quality of the robot is evaluated for each of the assigned locations. A first set of candidate grasp locations on the object is determined based on data representative of simulated grasp quality from the evaluation. Based on sensor data from an actual experiment for the robot grasping using each of the candidate grasp locations, an actual object grasp quality is evaluated for each of the candidate locations.

## Claims

### Claim 1

A method for training an object picking robot with real and simulated grasp performance data, comprising:
(a) assigning, by a processor, a plurality of grasp locations on an object based on known or estimated physical properties of the object;
(b) performing, by the processor, a first simulation experiment for the robot grasping the object using a first set of the plurality of assigned grasp locations;
(c) evaluating, by the processor and based on a first set of simulation data from the first simulation experiment, a simulated object grasp quality of the robot grasping for each of the first set of assigned grasp locations;
(d) determining, by the processor, a first set of candidate grasp locations on the object based on data representative of a simulated grasp quality obtained in (c);
(e) evaluating, by the processor and based on a first set of grasp quality sensor data from a first actual experiment for the robot grasping the object using each of the first set of candidate grasp locations, an actual object grasp quality of the robot grasping for the each of the first set of candidate grasp locations; and
(f) for the each of the first set of candidate grasp locations, determining, by the processor, a convergence of the actual object grasp quality and the simulated object grasp quality based on: the first set of grasp quality sensor data from the first actual experiment, and the data representative of the simulated grasp quality.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
(g) in response to determining, in (f), an absence of the convergence of the actual and the simulated object grasp quality for at least one of the first set of candidate grasp locations, determining, by the processor, a grasp success probability value for the each of the first set of candidate grasp locations based on the first set of grasp quality sensor data obtained in (e).

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
(h) assigning, by the processor, grasp success probability values respectively determined in (g) for the each of the first set of candidate grasp locations.

### Claim 4 (depends on 2)

The method of claim 2, wherein determining the grasp success probability value includes:
(I) transforming, by the processor, each of the first set of grasp quality sensor data into a discrete value; and
(II) scoring, by the processor, the each of the first set of candidate grasp locations based on respective discrete values from (I), wherein a score value for the each of the first set of candidate grasp locations is proportional to the grasp success probability value determined in (g).

### Claim 5 (depends on 4)

The method of claim 4, wherein the transforming includes determining the discrete value based on a plurality of grasp quality sensor readings for the each of the first set of candidate grasp locations.

### Claim 6 (depends on 1)

The method of claim 1, wherein (b)-(f) are performed in a machine learning training environment, and wherein the method further comprises:
(g) in response to determining, in (f), a presence of the convergence of the actual and the simulated object grasp quality for at least one of the first set of candidate grasp locations, monitoring, by the processor, a run time grasp quality of the robot grasping using the at least one candidate grasp location based on a run time set of grasp quality sensor data obtained from the robot grasping in a machine learning run time computing environment.

### Claim 7 (depends on 6)

The method of claim 6, further comprising:
(h) in response to the run time grasp quality monitored in (g) decreasing below or otherwise not meeting a user-predetermined quality threshold, iterating, by the processor, and for at least one iteration, through (b)-(f) for the at least one candidate grasp location.

### Claim 8

A system for training an object picking robot with real and simulated grasp performance data, comprising:
one or more memory devices; and one or more processors in communication with the one or more memory devices and the object picking robot, wherein the one or more processors are programmed to:
(a) assign a plurality of grasp locations on an object based on known or estimated physical properties of the object;
(b) perform a first simulation experiment for the robot grasping the object using a first set of the plurality of assigned grasp locations;
(c) evaluate, based on a first set of simulation data from the first simulation experiment, a simulated object grasp quality of the robot grasping for each of the first set of assigned grasp locations;
(d) determine a first set of candidate grasp locations on the object based on data representative of a simulated grasp quality obtained in (c);
(e) evaluate, based on a first set of sensor data from a first actual experiment for the robot grasping the object using each of the first set of candidate grasp locations, an actual object grasp quality of the robot grasping for the each of the first set of candidate grasp locations; and
(f) for the each of the first set of candidate grasp locations, determine a convergence of the actual object grasp quality and the simulated object grasp quality based on: the first set of grasp quality sensor data from the first actual experiment, and the data representative of the simulated grasp quality.

### Claim 9 (depends on 8)

The system of claim 8, wherein the one or more processors are further programmed to:
(g) in response to determining, in (f), an absence of the convergence of the actual and the simulated object grasp quality for at least one of the first set of candidate grasp locations, determine a grasp success probability value for the each of the first set of candidate grasp locations based on the first set of sensor data obtained in (e).

### Claim 10 (depends on 9)

The system of claim 9, wherein the one or more processors are further programmed to:
(h) further in response to determining, in (f), the absence of the convergence of the actual and the simulated object grasp quality for the at least one of the first set of candidate grasp locations, iterate, for at least one iteration, through (b)-(f) for the at least one candidate grasp location.

### Claim 11 (depends on 10)

The system of claim 10, wherein, for iterating through (b)-(f) for the at least one candidate grasp location for the at least one iteration, the one or more processors are further programmed to:
(I) determine a grasp success probability distribution for the first set of candidate grasp locations based on grasp success probability values determined in (g); and
(II) impose the grasp success probability distribution determined in (I) on the first set of candidate grasp locations to prepare at least a second simulation experiment for the robot grasping the object.

### Claim 12 (depends on 11)

The system of claim 11, wherein, for iterating through (b)-(f) for the at least one candidate grasp location for the at least one iteration, the one or more processors are further programmed to:
(III) determine at least a second set of the plurality of assigned grasp locations based on the grasp success probability distribution imposed in (II) on the at least a second simulation experiment; and
(IV) perform the at least a second simulation experiment using the at least a second set of assigned grasp locations.

### Claim 13 (depends on 12)

The system of claim 12, wherein for iterating through (b)-(f) for the at least one candidate grasp location for the at least one iteration, the one or more processors are further programmed to:
(V) determine a maximum log likelihood of success for each of the at least a second set of assigned grasp locations based on the grasp success probability distribution determined in (I), the maximum log likelihood of success including maximum log likelihood of success valves indicative of likely success of the at least a second set of assigned grasp locations; and
(VI) determine at least a second set of candidate grasp locations based on maximum log likelihood of success values respectively determined in (V) for the each of the at least a second set of assigned grasp locations.

### Claim 14 (depends on 13)

The system of claim 13, wherein for iterating through (b)-(f) for the at least one candidate grasp location for the at least one iteration, the one or more processors are further programmed to:
(VII) evaluate, based on at least a second set of simulation data from the at least a second simulation experiment, a simulated grasp quality of the robot grasping for the each of the at least a second set of assigned grasp locations on the object.

### Claim 15 (depends on 14)

The system of claim 14, wherein, for determining the maximum log likelihood of success, the one or more processors are further programmed to: determine the maximum log likelihood of success further based on data representative of the simulated grasp quality obtained in (VII).

### Claim 16 (depends on 10)

The system of claim 10, wherein the one or more processors are further programmed to:
(i) assign a hyper-parameter representative of a simulated grasp quality for at least a second simulation experiment for the robot grasping the object for tuning the machine learning model.

### Claim 17 (depends on 16)

The system of claim 16, wherein, for training a plurality of object picking robots including a first robot and at least a second robot, the one or more processors are further programmed to:
(j) share the hyper-model parameter assigned in (i) to a first robot with at least a second robot.

### Claim 18

A non-transitory computer-readable storage medium storing processor-executable instructions for training an object picking robot with real and simulated grasp performance data, wherein, when executed by one or more processors, the processor-executable instructions cause the one or more processors to:
(a) assign a plurality of grasp locations on an object based on known or estimated physical properties of the object;
(b) perform a first simulation experiment for the robot grasping the object using a first set of the plurality of assigned grasp locations;
(c) evaluate, based on a first set of simulation data from the first simulation experiment, a simulated object grasp quality of the robot grasping for each of the first set of assigned grasp locations;
(d) determine a first set of candidate grasp locations on the object based on data representative of a simulated grasp quality obtained in (c); and
(e) evaluate, based on a first set of grasp quality sensor data from a first actual experiment for the robot grasping the object using each of the first set of candidate grasp locations, an actual object grasp quality of the robot grasping for the each of the first set of candidate grasp locations, and
(f) for the each of the first set of candidate grasp locations, determine a convergence of the actual object grasp quality and the simulated object grasp quality based on: the first set of grasp quality sensor data from the first actual experiment, and the data representative of the simulated grasp quality.

### Claim 19 (depends on 18)

The non-transitory computer-readable storage medium of claim 18, wherein, when executed by the one or more processors, the processor-executable instructions further cause the one or more processors to:
(g) in response to determining, in (f), an absence of the convergence of the actual and the simulated object grasp quality for at least one of the first set of candidate grasp locations, determine a grasp success probability value for the each of the first set of candidate grasp locations based on the first set of grasp quality sensor data obtained in (e); and
(h) assign grasp success probability values respectively determined in (g) for the each of the first set of candidate grasp locations.