# US 11,839,977 B2 · Control apparatus, work robot, non-transitory computer-readable medium, and control method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,839,977 B2
- Document ID: US11839977B2
- Status: Granted
- Kind: B2
- Application number: 17/326,483
- Filed: 2021-05-21
- Published: 2021-09-02
- Granted: 2023-12-12
- Earliest claimed: 2018-10-15
- Assignees: MUJIN, Inc.
- Applicants: Mujin, Inc.
- Inventors: Rosen Diankov
- Web page: https://patentgrep.com/patent/US11839977B2

## CPC

- B25J 9/163: {learning, adaptive, model based, rule based expert control}
- B25J 9/1666: {Avoiding collision or forbidden zones}
- B25J 9/1679: {characterised by the tasks executed}
- G05B 2219/40317: For collision avoidance and detection
- G05B 2219/40475: In presence of moving obstacles, dynamic environment
- G05B 2219/40492: Model manipulator by spheres for collision avoidance
- G05B 2219/40516: Replanning
- G05B 2219/40521: Alternative, allowable path substitution if arm movements not possible
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)

## Abstract

A control apparatus for controlling operation of a work robot for performing work inside a target region using a manipulator includes a trajectory information acquiring unit for acquiring N−1 or N pieces of trajectory information respectively indicating N−1 or N trajectories connecting N work regions where the work robot performs a series of work operations in order of a series of work operations; a classifying unit for classifying the N−1 or N trajectories as (i) trajectories that need correction or (ii) trajectories that do not need correction; and a trajectory planning unit for planning a trajectory of a tip of the manipulator between two work regions relating to the each of the one or more trajectories, for each of the one or more trajectories classified as a trajectory that needs correction by the classifying unit.

## Claims

### Claim 1

A control apparatus comprising:
a non-transitory computer-readable storage medium configured, when the control apparatus is in communication with a robot having a manipulator for performing work inside a target region, to store information indicating at least one of a position, orientation, shape, or size of a first body within the target region, and to store trajectory information that represents a trajectory for movement of the manipulator within the target region based on the information indicating at least one of the position, orientation, shape, or size of the first body; and
a processor configured to, when the first body manipulated by the manipulator of the robot is arranged inside the target region, to:
acquire change information that indicates a change in the target region that occurred after calculation of the stored trajectory information, the change information including at least one of: (i) a change in an orientation or position of the first body in the target region, (ii) removal of the first body from the target region, or (iii) a new body to be manipulated by the manipulator of the robot being arranged or to be arranged inside the target region;
determine, based on the change information, whether the trajectory for movement of the manipulator needs correction due to the change in the target region; and
in response to a determination that the trajectory needs correction due to the change inside the target region as indicated by the change information, determine, based on the change information, a new trajectory for movement of the manipulator in the target region.

### Claim 2 (depends on 1)

The control apparatus of claim 1, wherein the change information indicates the change in the target region to include removal of the first body from the target region, such that the processor is configured to determine whether the trajectory for movement of the manipulator in the target region needs correction due to removal of the first body from the target region.

### Claim 3 (depends on 1)

The control apparatus of claim 1, wherein the change information indicates the change in the target region to include the new body being arranged or to be arranged inside the target region, such that the processor is configured to determine whether the trajectory for movement of the manipulator in the target region needs correction due to the new body being arranged or to be arranged inside the target region.

### Claim 4 (depends on 1)

The control apparatus of claim 1, wherein the change information indicates the change in the target region to include the change in the orientation of the first body, such that the processor is configured to determine whether the trajectory for movement of the manipulator needs correction due to the change in the orientation of the first body.

### Claim 5 (depends on 1)

The control apparatus of claim 1, wherein the processor is configured to determine whether the trajectory needs correction by determining whether the first body or the new body would cause interference to the trajectory, wherein interference to the trajectory occurs when a distance between the manipulator and the first body or the new body in the target region becomes less than a predetermined value.

### Claim 6 (depends on 5)

The control apparatus of claim 5, wherein the processor is configured to determine the new trajectory by replanning the trajectory at one or more locations where the first body or the new body would cause interference to the trajectory.

### Claim 7 (depends on 6)

The control apparatus of claim 6, wherein the processor is configured to replan the trajectory only at the one or more locations where the first body or the new body would cause interference to the trajectory.

### Claim 8 (depends on 1)

The control apparatus of claim 1, wherein the processor is configured to determine whether the trajectory needs correction based on a model of the manipulator and based on a model of the first body or of the new body.

### Claim 9 (depends on 8)

The control apparatus of claim 8, wherein the model of the manipulator and the model of the first body or the new body are each formed from at least one of a spherical shape, a cylindrical shape, or a polygonal shape.

### Claim 10 (depends on 1)

The control apparatus of claim 1, wherein the trajectory information represents a plurality of unit trajectories which connect a series of locations at which the manipulator is to perform a series of operations, wherein the plurality of unit trajectories form an overall trajectory, and wherein the trajectory is one unit trajectory of the plurality of unit trajectories, and
wherein the processor is configured to determine a new overall trajectory due to the change in the target region by replanning the one unit trajectory while reusing other unit trajectories of the plurality of unit trajectories.

### Claim 11 (depends on 1)

The control apparatus of claim 1, wherein the processor is configured to control movement of the robot based on the new trajectory.

### Claim 12

A non-transitory computer-readable medium having instructions that, when executed by a processor, causes the processor to:
acquire, when a first body manipulated by a manipulator of a robot is arranged inside a target region, and the processor is in communication with the robot having the manipulator for performing work inside the target region, information indicating at least one of a position, orientation, shape, or size of the first body within the target region, and trajectory information that represents a trajectory for movement of the manipulator within the target region based on the information indicating at least one of the position, orientation, shape, or size of the first body;
acquire change information that occurred after calculation of the trajectory information, the change information including at least one of: (i) a change in an orientation or position of a first body to be manipulated by the manipulator of the robot in the target region, (ii) removal of the first body from the target region, or (iii) a new body to be manipulated by the manipulator of the robot being arranged or to be arranged inside the target region;
determine, based on the change information, whether the trajectory for movement of the manipulator needs correction due to the change in the target region; and
in response to a determination that the trajectory needs correction due to the change inside the target region as indicated by the change information, determine, based on the change information, a new trajectory for movement of the manipulator in the target region.

### Claim 13 (depends on 12)

The non-transitory computer-readable medium of claim 12, wherein the change information indicates the change in the target region to include removal of the first body from the target region, such that the instructions, when executed by the processor, cause the processor to determine whether the trajectory for movement of the manipulator in the target region needs correction due to removal of the first body from the target region.

### Claim 14 (depends on 12)

The non-transitory computer-readable medium of claim 12, wherein the change information indicates the change in the target region to include the new body being arranged or to be arranged inside the target region, such that the instructions, when executed by the processor, cause the processor to determine whether the trajectory for movement of the manipulator in the target region needs correction due to the new body being arranged or to be arranged inside the target region.

### Claim 15 (depends on 12)

The non-transitory computer-readable medium of claim 12, wherein the change information indicates the change in the target region to include the change in the orientation of the first body, such that the instructions, when executed by the processor, cause the processor to determine whether the trajectory for movement of the manipulator needs correction due to the change in the orientation of the first body.

### Claim 16 (depends on 12)

The non-transitory computer-readable medium of claim 12, wherein the instructions, when executed by the processor, cause the processor to determine whether the trajectory needs correction by determining whether the first body or the new body would cause interference to the trajectory, wherein interference to the trajectory occurs when a distance between the manipulator and the first body or the new body in the target region becomes less than a predetermined value.

### Claim 17 (depends on 16)

The non-transitory computer-readable medium of claim 16, wherein the instructions, when executed by the processor, cause the processor to determine the new trajectory by replanning the trajectory at one or more locations where the first body or the new body would cause interference to the trajectory.

### Claim 18 (depends on 17)

The non-transitory computer-readable medium of claim 17, wherein the instructions, when executed by the processor, cause the processor to replan the trajectory only at the one or more locations where the first body or the new body would cause interference to the trajectory.

### Claim 19

A control apparatus comprising:
a processor configured, when the control apparatus is in communication with a robot having a manipulator for manipulating a first body inside a work point of a target region, to:
determine, as a first model, a model of the manipulator, wherein the first model has dimensions that are larger than the manipulator;
determine, as a second model, a model of a second body in the target region that is an obstacle to be avoided by the manipulator as the manipulator traverses a unit trajectory towards the work point, wherein the second model has dimensions that are larger than the second body;
performing collision determination to determine whether there would be a collision between the first model and the second model; and
controlling movement of the robot based on a result of the collision determination towards the work point to begin manipulating the first body with the manipulator.