# US 2020/0401801 A1 · Monitor device provided with camera for capturing motion image of motion of robot device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2020/0401801 A1 - Document ID: US20200401801A1 - Status: Published application - Kind: A1 - Application number: 16/897,279 - Filed: 2020-06-10 - Published: 2020-12-24 - Earliest claimed: 2019-06-21 - Assignees: Fanuc Corporation - Applicants: Fanuc Corporation - Inventors: Masafumi OOBA - Web page: https://patentgrep.com/patent/US20200401801A1 ## CPC - G06T 7/001: {using an image reference approach} - G06V 20/10: Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) - G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) - G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments - H04N 5/76: Television signal recording - H04N 7/18: Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast - H04N 7/183: {for receiving images from a single remote source} - G06T 2207/30164: Workpiece; Machine component - G06T 2207/30241: Trajectory - G06T 2207/30244: Camera pose - H04N 5/772: {the recording apparatus and the television camera being placed in the same enclosure} - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A monitor device includes a camera for capturing a motion image of a motion of a robot device and an acceleration sensor for detecting an operational status of the robot device. A storage control unit for a robot controller performs control for storing, in a storage unit, a motion image captured by the camera and attached with a time, and control for storing, in the storage unit, an acceleration acquired from an acceleration sensor and attached with a time. When the operational status of the robot device deviates from a reference operational status, an extraction unit extracts, from the storage unit, a motion image in a period preceding the deviation from the reference operational status and stores the extracted motion image in the storage unit. ## Claims ### Claim 1 A monitor device for monitoring a motion of a robot device including a robot and an operation tool attached to the robot, comprising: a camera for capturing a motion image of a motion of the robot device; a status detector for detecting an operational status of the robot device; and an arithmetic processing device for acquiring the motion image captured by the camera and an output of the status detector; wherein the arithmetic processing device including: a storage unit for storing predetermined information; a storage control unit for operating the information stored in the storage unit; an extraction unit for extracting part of the information stored in the storage unit; and a determination unit for determining the operational status of the robot device, the storage control unit performs control for storing a motion image in the storage unit, the image being captured by the camera and attached with a time or an elapsed time from a predetermined reference time, control for storing a variable number in the storage unit, the variable number being acquired from the output of the status detector and attached with the time or the elapsed time, and control for deleting a motion image captured before a predetermined period preceding a current time, the determination unit determines whether the operational status of the robot device deviates from a predetermined reference operational status, and if the determination unit determines that the operational status of the robot device deviates from the reference operational status, the extraction unit extracts, from the storage unit, a motion image captured in a predetermined period including a period preceding the deviation from the reference operational status and stores an extracted motion image in the storage unit. ### Claim 2 (depends on 1) The monitor device according to claim 1, wherein the camera is disposed so as to capture an image of a state in which the operation tool performs an operation to the workpiece. ### Claim 3 (depends on 1) The monitor device according to claim 1, wherein if the determination unit determines that the operational status of the robot device deviates from the reference operational status, the extraction unit extracts a motion image captured in a period subsequent to the deviation from the reference operational status and stores an extracted motion image in the storage unit. ### Claim 4 (depends on 1) The monitor device according to claim 1, wherein the arithmetic processing device includes a motion control unit for transmitting a motion command of the robot at every predetermined control period, and the storage control unit stores, in the storage unit, variable numbers acquired from the output of the status detector at the same time intervals as a control period of the robot. ### Claim 5 (depends on 1) The monitor device according to claim 1, wherein the variable numbers acquired from the output of the status detector include at least one of an acceleration of the robot and a position of the robot. ### Claim 6 (depends on 1) The monitor device according to claim 1, wherein the storage control unit stores, in the storage unit, at least one of information on a warning of an operation of the robot device, information on a command statement of a motion program corresponding to a motion of the robot device, and information on interpolation points generated between movement points described in the motion program, the information being attached with a time or the elapsed time. ### Claim 7 (depends on 4) The monitor device according to claim 4, wherein the arithmetic processing device includes a prediction unit for predicting a motion of the robot, the prediction unit estimates a position of the robot after driving the robot by a subsequent motion command based on the variable number acquired from the output of the status detector and stored in the storage unit, the determination unit determines whether the robot position estimated by the prediction unit is reachable by the robot, and if the robot position estimated by the prediction unit is not reachable by the robot, the robot is stopped before being driven by the subsequent motion command.