# US 2023/0069393 A1 · Control device, control method and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0069393 A1
- Document ID: US20230069393A1
- Status: Published application
- Kind: A1
- Application number: 17/799,714
- Filed: 2020-02-25
- Published: 2023-03-02
- Earliest claimed: 2020-02-25
- Assignees: NEC Corporation
- Applicants: NEC Corporation
- Inventors: Masatsugu OGAWA; Hisaya Wakayame; Takehiro Itou; Mineto Satoh; Hiroyuki Oyama; Nobuharu Kami
- Web page: https://patentgrep.com/patent/US20230069393A1

## CPC

- B25J 9/1661: {characterised by task planning, object-oriented languages}
- B25J 9/1674: {characterised by safety, monitoring, diagnostic}
- B25J 13/08: by means of sensing devices, e.g. viewing or touching devices
- G05B 19/4155: characterised by program execution, i.e. part program or machine function execution, e.g. selection of a program
- G05B 2219/39091: Avoid collision with moving obstacles
- G05B 2219/40202: Human robot coexistence
- G05B 2219/40607: Fixed camera to observe workspace, object, workpiece, global
- G05B 2219/50391: Robot
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)

## Abstract

A control device 1C mainly includes an operation sequence generation means 16C and a synchronization management means 17C. The operation sequence generation means 16C is configured to generate, based on an operation prediction result R2a of another working body which performs cooperative work with a robot which executes a task, an operation sequence Sra to be executed by the robot. The synchronization management means 17C is configured to synchronize an operation executed by the robot during execution of the operation sequence Sra and an operation executed by the other working body.

## Claims

### Claim 1

A control device comprising:
at least one memory configured to store instructions; and
at least one processor configured to execute the instructions to generate,
based on an operation prediction result of another working body which performs cooperative work with a robot which executes a task,
an operation sequence to be executed by the robot; and synchronize an operation executed by the robot during execution of the operation sequence and an operation executed by the other working body.

### Claim 2 (depends on 1)

The control device according to claim 1,
wherein, in a case that the operation executed by the robot and the operation executed by the other working body are not synchronized, the at least one processor is configured to execute the instructions to adjust an execution timing of an operation included in the operation sequence to be executed by the robot.

### Claim 3 (depends on 1)

The control device according to claim 1,
wherein, in a case that the operation executed by the robot and the operation executed by the other working body are not synchronized, the at least one processor is configured to execute the instructions to regenerate the operation sequence.

### Claim 4 (depends on 3)

The control device according to claim 3,
wherein,
in a case that the task cannot be completed regardless of an adjustment of an execution timing of an operation included in the operation sequence to be executed by the robot, or
in a case that a completion of the task on an assumption that the regenerated operation sequence is executed is predicted to be earlier than the completion of the task on an assumption that the execution timing of an operation included in the operation sequence to be executed by the robot is adjusted,
the at least one processor is configured to execute the instructions to cause the robot to execute the regenerated operation sequence.

### Claim 5 (depends on 1)

The control device according to claim 1,
wherein the at least one processor is configured to further execute the instructions to predict the operation to be executed by the other moving body based on at least one of
the output signal from a detection device whose detection range includes a workspace where the task is executed or
other working body operation related information outputted by the other working body, and
wherein the at least one processor is configured to execute the instructions to generate the operation sequence based on a result of predicting the operation to be executed by the other working body.

### Claim 6 (depends on 1)

The control device according to claim 1,
wherein the at least one processor is configured to execute the instructions to observe a state of objects in the workspace, and
wherein the at least one processor is configured to execute the instructions to generate the operation sequence based on the operation prediction result and an observation result regarding the state of the objects.

### Claim 7 (depends on 1)

The control device according to claim 1,
wherein the at least one processor is configured to execute the instructions to give a notification regarding an operation to be executed by the robot to the other working body that is a worker.

### Claim 8 (depends on 7)

The control device according to claim 7,
wherein the at least one processor is configured to execute the instructions to display or output by audio, as the notification, information on a future operation to be executed in a predetermined time by the robot.

### Claim 9 (depends on 7)

The control device according to claim 7,
wherein the at least one processor is configured to execute the instructions to give the notification by causing the robot to execute a predetermined gesture.

### Claim 10 (depends on 9)

The control device according to claim 9,
wherein the at least one processor is configured to execute the instructions to generate the operation sequence including the gesture for giving the notification.

### Claim 11 (depends on 1)

The control device according to claim 1, further comprising
wherein the at least one processor is configured to execute the instructions to detect an intention of the other working body that is a worker,
wherein the at least one processor is configured to execute the instructions, based on the intention, to regenerate the operation sequence or
adjust an execution timing of an operation included in the operation sequence to be executed by the robot.

### Claim 12 (depends on 1)

The control device according to claim 1,
wherein the at least one processor is configured to execute the instructions to convert an objective task, which is a task to be performed by the robot, into a logical formula based on a temporal logic;
to generate, from the logical formula, a time step logical formula that is a logical formula representing states at each time step for completing the objective task; and
to generate, based on the time step logical formula, the operation sequence that is a sequence of subtasks to be executed by the robot.

### Claim 13

A control method executed by a computer, the control method comprising:
generating,
based on an operation prediction result of another working body which performs cooperative work with a robot which executes a task,
an operation sequence to be executed by the robot; and
synchronizing an operation executed by the robot during execution of the operation sequence and an operation executed by the other working body.

### Claim 14

A non-transitory computer readable storage medium storing a program executed by a computer, the program causing the computer to:
generate,
based on an operation prediction result of another working body which performs cooperative work with a robot which executes a task,
an operation sequence to be executed by the robot; and
synchronize an operation executed by the robot during execution of the operation sequence and an operation executed by the other working body.
