# US 2024/0424675 A1 · Working State Determination Device, Method, and Program

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0424675 A1
- Document ID: US20240424675A1
- Status: Published application
- Kind: A1
- Application number: 18/289,277
- Filed: 2022-03-28
- Published: 2024-12-26
- Earliest claimed: 2021-05-12
- Assignees: OMRON Corporation
- Applicants: OMRON Corporation
- Inventors: Kentaro Oki; Shunsuke Matoba
- Web page: https://patentgrep.com/patent/US20240424675A1

## CPC

- B25J 9/1674: {characterised by safety, monitoring, diagnostic}
- B25J 9/1612: {characterised by the hand, wrist, grip control}
- B25J 9/163: {learning, adaptive, model based, rule based expert control}
- B25J 9/1687: {Assembly, peg and hole, palletising, straight line, weaving pattern movement}
- B25J 13/085: {Force or torque sensors (B25J13/082, B25J13/084 take precedence)}
- G05B 2219/39529: Force, torque sensor in wrist, end effector
- G05B 2219/40032: Peg and hole insertion, mating and joining, remote center compliance
- G05B 2219/40087: Align hand on workpiece to pick up workpiece, peg and hole
- G05B 2219/40541: Identification of contact formation, state from several force measurements
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)

## Abstract

An acquisition unit (32) acquires data detected by a force sensor that detects a force acting on an hand tip portion of a robot hand, and a determination unit (38) determines a working state of the robot hand by inputting data acquired by the acquisition unit (32) to a determiner 36 that determines the working state of the robot hand and in which a first determiner that extracts a feature of a normal working state and a second determiner that extracts a feature of an abnormal working state are coupled, the determiner (36) outputting three states as the working state, the three states being a work completion state, a work continuation state, and a work continuation impossible state.

## Claims

### Claim 1

A working state determination device, comprising:
an acquisition unit configured to acquire data detected by a force sensor that detects a force acting on a hand tip portion of a robot hand; and
a determination unit configured to determine a working state of the robot hand by inputting data acquired by the acquisition unit to a determiner that determines the working state of the robot hand and in which a first determiner that extracts a feature of a normal working state and a second determiner that extracts a feature of an abnormal working state are coupled, the determiner outputting three states as the working state, the three states being a work completion state, a work continuation state, and a work continuation impossible state.

### Claim 2 (depends on 1)

The working state determination device according to claim 1, wherein time-series data acquired by the acquisition unit is input to the determiner, and a time duration of data input to the second determiner is longer than a time duration of data input to the first determiner.

### Claim 3 (depends on 2)

The working state determination device according to claim 2, wherein the time duration for the first determiner is a time duration corresponding to a transient response of data generated at a time of work completion, and the time duration for the second determiner is a time duration corresponding to a time until an abnormality occurs and an operation of the robot hand is stopped.

### Claim 4 (depends on 1)

The working state determination device according to claim 1, wherein the first determiner is learned by using learning data in which each piece of data before transition of time-series data in a case in which a working state transitions from work continuation to work completion is labeled as work continuation and data after transition is labeled as work completion, and the second determiner is learned by using learning data in which data before transition of the time-series data in a case in which a working state transitions from work continuation to work continuation impossible is labeled as work continuation and data after transition is labeled as work continuation impossible.

### Claim 5 (depends on 4)

The working state determination device according to claim 4, further comprising a learning unit configured to execute learning of the determiner by using the learning data.

### Claim 6 (depends on 5)

The working state determination device according to claim 5, wherein the learning unit is configured to generate pseudo learning data using the learning data and a random number, add the pseudo learning data to the learning data, and use the pseudo learning data for learning of the determiner.

### Claim 7 (depends on 1)

The working state determination device according to claim 1, wherein:
the acquisition unit is configured to further acquire a position of the hand tip portion of the robot hand, and
the determination unit is configured to determine the working state by inputting data detected by the force sensor and the position of the hand tip portion to the determiner.

### Claim 8 (depends on 1)

The working state determination device according to claim 1, wherein the determiner includes a first neural network constituting the first determiner and a second neural network constituting the second determiner, and outputs a result obtained by combining an output of the first neural network and an output of the second neural network.

### Claim 9 (depends on 1)

The working state determination device according to claim 1, further comprising a control unit configured to control operation of the robot hand based on the working state determined by the determination unit.

### Claim 10

A working state determination method, wherein:
an acquisition unit is configured to acquire data detected by a force sensor that detects a force acting on an hand tip portion of a robot hand; and
a determination unit is configured to determine a working state of the robot hand by inputting data acquired by the acquisition unit to a determiner that determines the working state of the robot hand and in which a first determiner that extracts a feature of a normal working state and a second determiner that extracts a feature of an abnormal working state are coupled, the determiner outputting three states as the working state, the three states being a work completion state, a work continuation state, and a work continuation impossible state.

### Claim 11

A non-transitory storage medium storing a working state determination program executable by a computer to function as:
an acquisition unit configured to acquire data detected by a force sensor that detects a force acting on an hand tip portion of a robot hand, and
a determination unit configured to determine a working state of the robot hand by inputting data acquired by the acquisition unit to a determiner that determines the working state of the robot hand and in which a first determiner that extracts a feature of a normal working state and a second determiner that extracts a feature of an abnormal working stats are coupled, the determiner outputting three states as the working state, the three states being a work completion state, a work continuation state, and a work continuation impossible state.
