# US 2023/0418251 A1 · Control system and action generation method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0418251 A1
- Document ID: US20230418251A1
- Status: Published application
- Kind: A1
- Application number: 18/112,659
- Filed: 2023-02-22
- Published: 2023-12-28
- Earliest claimed: 2022-06-23
- Assignees: Hitachi, Ltd.
- Applicants: Hitachi, Ltd.
- Inventors: Kanako ESAKI; Tadayuki MATSUMURA; Hiroyuki MIZUNO
- Web page: https://patentgrep.com/patent/US20230418251A1

## CPC

- G05B 13/048: {using a predictor}
- B25J 9/1628: {characterised by the control loop}
- G05B 2219/45063: Pick and place manipulator
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})

## Abstract

To act appropriately in consideration of uncertainty of a surrounding environment, it is provided a control system for generating an action for controlling a controlled device, comprising: a reception module configured to receive sensor data acquired by observing a state of a surrounding environment of the controlled device; a self-recognition module configured to derive, through use of a self-recognition prediction model that predicts a self-range being a range having a predictability and a controllability relating to the controlled device, a self-recognition block that defines the self-range from the sensor data; a target action prediction module configured to derive, through use of a target action prediction model that predicts a target action of the controlled device, the target action from the sensor data; and a switching module configured to select one of the self-recognition block or the target action in order to generate an action of the controlled device.

## Claims

### Claim 1

A control system for generating an action for controlling a controlled device, comprising:
a reception module configured to receive sensor data acquired by observing a state of a surrounding environment of the controlled device;
a self-recognition module configured to derive, through use of a self-recognition prediction model that predicts a self-range being a range having a predictability and a controllability relating to the controlled device, a self-recognition block that defines the self-range from the sensor data;
a target action prediction module configured to derive, through use of a target action prediction model that predicts a target action of the controlled device, the target action from the sensor data; and
a switching module configured to select one of the self-recognition block or the target action in order to generate an action of the controlled device.

### Claim 2 (depends on 1)

The control system according to claim 1,
wherein the predictability means being able to predict a change in a shape and a motion of a target object controlled by the controlled device, and
wherein the controllability means changing the shape and the motion through the action of the controlled device.

### Claim 3 (depends on 1)

The control system according to claim 1, wherein the switching module is configured to select the self-recognition block in a case where a size of the self-recognition block is larger than a sum of a size of a target object controlled by the controlled device and a size of a predetermined surrounding region.

### Claim 4 (depends on 1)

The control system according to claim 1, wherein the switching module is configured to select the self-recognition block in a case where an estimated execution time derived by the target action prediction module is longer than a predetermined threshold value.

### Claim 5 (depends on 4)

The control system according to claim 4, further comprising an action generation module configured to generate, through use of an action generation model, an action from one of the self-recognition block or the target action selected by the switching module,
wherein in a case where the self-recognition block is selected, the action generation module is configured to abandon an original action relating to a target object controlled by the controlled device, and to generate an action of increasing precision of the self-recognition block.

### Claim 6 (depends on 1)

The control system according to claim 1, wherein the switching module is configured to select the self-recognition block in a case where a current time is before an action start target time.

### Claim 7 (depends on 6)

The control system according to claim 6, further comprising an action generation module configured to generate, through use of an action generation model, an action from one of the self-recognition block or the target action selected by the switching module,
wherein the action generation module is configured to generate an action of increasing precision of the self-recognition block until the action start target time.

### Claim 8 (depends on 1)

The control system according to claim 1, further comprising an action generation module configured to generate, through use of an action generation model, from one of the self-recognition block or the target action selected by the switching module, an action in which the controlled device grips a gripping target object and moves the gripping target object.

### Claim 9 (depends on 1)

The control system according to claim 1, further comprising an action generation module configured to generate, through use of an action generation model, from one of the self-recognition block or the target action selected by the switching module, an action in which the controlled device guides a person such that the controlled device does not interfere with the person.

### Claim 10 (depends on 1)

The control system according to claim 1,
wherein the self-recognition module is configured to update the self-recognition prediction model through use of the sensor data, and
wherein the target action prediction module is configured to update the target action prediction model through use of the sensor data.

### Claim 11

An action generation method, which is executed by a control system for generating an action for controlling a controlled device,
the control system including a calculation device configured to execute predetermined calculation processing, and a storage device coupled to the calculation device,
the action generation method comprising:
a reception step of receiving, by the calculation device, sensor data acquired by observing a state of a surrounding environment of the controlled device;
a self-recognition step of deriving, by the calculation device, through use of a self-recognition prediction model that predicts a self-range being a range having a predictability and a controllability relating to the controlled device, a self-recognition block that defines the self-range from the sensor data;
a target action prediction step of deriving, by the calculation device, through use of a target action prediction model that predicts a target action of the controlled device, the target action from the sensor data; and
a switching step of selecting, by the calculation device, one of the self-recognition block or the target action in order to generate an action of the controlled device.
