# US 2024/0173857 A1 · System and method for controlling the robot, electronic device and computer readable medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0173857 A1
- Document ID: US20240173857A1
- Status: Published application
- Kind: A1
- Application number: 17/758,962
- Filed: 2020-01-22
- Published: 2024-05-30
- Earliest claimed: 2020-01-22
- Assignees: ABB Schweiz AG
- Applicants: ABB Schweiz AG
- Inventors: Yichao Mao; Zheng Ju; Xiaohao Chen
- Web page: https://patentgrep.com/patent/US20240173857A1

## CPC

- B25J 9/163: {learning, adaptive, model based, rule based expert control}
- G05B 19/042: using digital processors (G05B19/05 takes precedence)
- B25J 9/1664: {characterised by motion, path, trajectory planning}
- B25J 9/1697: {Vision controlled systems}
- G05B 2219/39206: Joint space position control
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)

## Abstract

A system and a method for controlling the robot. The method includes determining, based on feedback data received from the robot, first position information of a joint of the robot. The feedback data indicates a movement of the joint. The method also includes determining second position information of the joint based on sensing data received from a sensor. The sensing data indicates a relative movement between the joint and a second object to be aligned with the first object. The method also predicts a target position for aligning the first object with the second object.

## Claims

### Claim 1

A method for controlling a robot comprising:
determining, based on feedback data received from the robot, first position information of a joint of the robot, the feedback data indicating a movement of the joint to which a tool being attached for holding a first object;
determining second position information of the joint based on sensing data received from a sensor, the sensing data indicating a relative movement between the joint and a second object to be aligned with the first object; and
generating, based on the first and the second position information, a prediction of a target position for aligning the first object with the second object.

### Claim 2 (depends on 1)

The method of claim 1, wherein determining the first position information comprises:
receiving the feedback data from the robot in a first time period;
obtaining, from the feedback data, a set of coordinate parameters of the robot in a first coordinate system of the robot; and
determining the first position information based on the set of coordinate parameters.

### Claim 3 (depends on 1)

The method of claim 1, wherein determining the second position information comprises:
receiving the sensing data from the sensor in a second time period, the second time period at least partially overlapping with a first time period for receiving the feedback data;
obtaining, from the sensing data, a set of position relationships between the first object and the second object in a second coordinate system of the sensor; and
determining the second position information based on the first set of position relationships.

### Claim 4 (depends on 1)

The method of claim 1, wherein generating the prediction comprises:
obtaining, from the first position information, a first set of sampled parameters characterizing a reference position of the joint at a predetermined time point;
obtaining, from the second position information, a second set of sampled parameters characterizing a reference position relationship between the joint of the robot and the second object at the predetermined time point; and
generating the prediction of the target position by fusing the first set of sampled parameters and the second set of sampled parameters based on a predetermined fusion mode associated with an expected operation to be performed by the robot.

### Claim 5 (depends on 4)

The method of claim 4, wherein the predetermined fusion mode comprises at least one of the following:
a predictor mode,
a filter mode,
a sum mode, and
a subtraction mode.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
generating a command for controlling the robot at least partially based on the prediction.

### Claim 7 (depends on 6)

The method of claim 6, wherein generating the command comprises:
determining a set of recording time points in a third time period for recording the feedback data, a start time point of a first time period for receiving the feedback data deviating from a start time point of the third time period by a predetermine time delay;
determining, based on the prediction, a predicted time point when the first object reaches the target position;
determining a predicted trajectory through which the first object moves to the target position based on the set of recording time points, the feedback data, the predicted time point and the prediction of the target position; and
generating the command based on the predicted trajectory.

### Claim 8

A system for controlling a robot comprising:
a data collector coupled to the robot and configured to determine, based on feedback data received from the robot, first position information of a joint of the robot, the feedback data indicating a movement of the joint to which a tool being attached for holding a first object, coupled to a sensor and configured to determine second position information of the joint based on sensing data received from the sensor, the sensing data indicating a relative movement between the joint and a second object to be aligned with the first object; and
a first estimator coupled to the data collector and configured to generate, based on the first and the second position information, a prediction of a target position aligning the first object with the second object.

### Claim 9 (depends on 8)

The system of claim 8, wherein the data collector is further configured to:
receive the feedback data from the robot in a first time period;
obtain, from the feedback data, a set of coordinate parameters of the robot in a first coordinate system of the robot; and
determine the first position information based on the set of coordinate parameters.

### Claim 10 (depends on 8)

The system of claim 8, wherein the data collector is further configured to:
receive the sensing data from the sensor in a second time period, the second time period at least partially overlapping with a first time period for receiving the feedback data;
obtain, from the sensing data, a set of position relationships between the first object and the second object in a second coordinate system of the sensor; and
determine the second position information based on the first set of position relationships.

### Claim 11 (depends on 8)

The system of claim 8, wherein the first estimator is further configured to:
obtain, from the first position information, a first set of sampled parameters characterizing a reference position of the joint at a predetermined time point;
obtain, from the second position information, a second set of sampled parameters characterizing a reference position relationship between the joint of the robot and the second object at the predetermined time point; and
generate the prediction of the target position by fusing the first set of sampled parameters and the second set of sampled parameters based on a predetermined fusion mode associated with an expected operation to be performed by the robot.

### Claim 12 (depends on 11)

The system of claim 11, wherein the predetermined fusion mode comprises at least one of the following:
a predictor mode,
a filter mode,
a sum mode, and
a subtraction mode.

### Claim 13 (depends on 8)

The system of claim 8, further comprising:
a command generator coupled to the first estimator and configured to generate the command for controlling the robot at least partially based on the prediction.

### Claim 14 (depends on 8)

The system of claim 8, wherein the command generator is further configured to:
determine a set of recording time points in a third time period for recording the feedback data, a start time point of a first time period for receiving the feedback data deviating from a start time point of the third time period by a predetermine time delay;
determine, based on the prediction, a predicted time point when the first object (131) reaches the target position;
determine a predicted trajectory through which the first object (131) moves to the target position based on the set of recording time points, the feedback data, the predicted time point and the prediction of the target position; and
generate the command based on the predicted trajectory.

### Claim 15 (depends on 1)

An electronic device comprising:
a processor; and
a memory coupled to the processor and storing instructions for execution, the instructions, when executed by the processor, causing the device to perform the method of claim 1.

### Claim 16 (depends on 1)

A computer readable medium comprising program instructions for causing an electronic device to perform at least the method of claim 1.
