# US 2023/0241779 A1 · State monitoring device and state monitoring method for industrial machinery

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0241779 A1
- Document ID: US20230241779A1
- Status: Published application
- Kind: A1
- Application number: 18/003,261
- Filed: 2021-07-01
- Published: 2023-08-03
- Earliest claimed: 2020-07-03
- Assignees: Fanuc Corporation
- Applicants: Fanuc Corporation
- Inventors: Naruki SHINOHARA
- Web page: https://patentgrep.com/patent/US20230241779A1

## CPC

- B25J 9/1697: {Vision controlled systems}
- G05B 19/0423: {Input/output}
- B25J 9/1656: {characterised by programming, planning systems for manipulators}
- B25J 9/1674: {characterised by safety, monitoring, diagnostic}
- G05B 2219/24097: Camera monitors controlled machine
- G05B 2219/31447: Process error event detection and continuous process image detection, storage
- G05B 2219/37572: Camera, tv, vision
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)

## Abstract

A robot state monitoring device 2 comprises: a camera which captures an image of a robot 3 under the control of a controller; a moving image generation unit which associates video data of the robot 3 acquired by the camera with input/output signals DO[1], AO[1], DI[1], and AI[1] of the controller along a time axis 830, and generates a moving image showing a state change of the robot 3 and the input/output signals DO[1], AO[1], DI[1], and AI[1]; and a moving image playback device which plays back the moving image generated by the moving image generation unit The moving image generation unit acquires the values of the input/output signals DO[1], AO[1], DI[1], and AI[1] at the recording time of each frame under the same cycle as the frame rate of the video data.

## Claims

### Claim 1

A state monitoring apparatus for an industrial machine, comprising:
an imaging device that captures an image of the industrial machine under control of a controller;
a video creation unit that associates a video data acquired by the imaging device with an input/output signal, which is at least one of an input or output signal of the controller, along a time axis, thereby creating a video showing a change in state of the industrial machine and a change in state of the input/output signal; and
a video player that plays the video created by the video creation unit.

### Claim 2 (depends on 1)

The state monitoring apparatus for the industrial machine according to claim 1, wherein the video creation unit acquires values of the input/output signal at times of recording frames in the same cycle as a frame rate of the video data.

### Claim 3 (depends on 1)

The state monitoring apparatus for the industrial machine according to claim 1, wherein the video creation unit creates a video including
a moving image display field that shows moving images of the industrial machine and
a timeline field that shows still images of the industrial machine in the frames arranged along the time axis and a time chart image of the input/output signal along the time axis.

### Claim 4 (depends on 3)

The state monitoring apparatus for the industrial machine according to claim 3, further comprising:
a program storage that stores a program for setting a value of the output signal of the controller; and
a program editing unit that edits the program based on an operation by an operator on an edition image including the timeline field.

### Claim 5 (depends on 1)

The state monitoring apparatus for the industrial machine according to claim 1, wherein the video player is a teaching device for the industrial machine.

### Claim 6

A state monitoring method for an industrial machine, comprising:
capturing an image of the industrial machine under control of a controller using an imaging device;
associating a video data acquired by the imaging device with an input/output signal, which is at least one of an input or output signal of the controller, along a time axis, thereby creating a video showing a change in state of the industrial machine and a change in state of the input/output signal; and
playing the video on a video player.
