# US 12,358,151 B2 · Robot control system and control device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,358,151 B2
- Document ID: US12358151B2
- Status: Granted
- Kind: B2
- Application number: 18/246,498
- Filed: 2022-03-04
- Published: 2023-10-26
- Granted: 2025-07-15
- Earliest claimed: 2021-03-31
- Assignees: JOHNAN Corporation
- Applicants: JOHNAN Corporation
- Inventors: Kozo Moriyama; Shin Kameyama; Truong Gia Vu; Lucas Brooks
- Web page: https://patentgrep.com/patent/US12358151B2

## CPC

- B25J 9/1697: {Vision controlled systems}
- B25J 13/08: by means of sensing devices, e.g. viewing or touching devices
- G05B 2219/39039: Two cameras detect same reference on workpiece to define its position in space
- G05B 2219/39057: Hand eye calibration, eye, camera on hand, end effector
- G05B 2219/40202: Human robot coexistence
- G05B 2219/40607: Fixed camera to observe workspace, object, workpiece, global
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)

## Abstract

Provided is a robot control system (1) including a robot (200), at least one first camera (300) attached or fixed to the robot, a second camera (400) attached to a worker, and a control device (100). The control device controls the robot by specifying a position of a workpiece W or a component, or a skeletal posture or states of a worker based on an image of the first camera and an image of the second camera.

## Claims

### Claim 1

A robot control system, comprising:
a robot;
at least one first camera attached or fixed to the robot;
a second camera attached to a worker; and
a control device configured to:
control the robot, when part of a work target is blocked from the at least one first camera due to the worker's body, based on an image from the at least one first camera and an image from the second camera, by:
specifying a position of a workpiece or a component,
specifying a skeletal posture of the worker, or
specifying states of the worker;
or
control the robot, when an entirety of the work target appears in an image from the at least one first camera, based on an image from the at least one first camera, by:
specifying a position of the workpiece or the component,
specifying the skeletal posture of the worker, or
specifying states of the worker.

### Claim 2

A robot control system, comprising:
a robot;
at least one first camera attached or fixed to the robot;
a second camera attached to a worker,
wherein the second camera includes an acceleration sensor; and
a control device, configured to:
control the robot, when a change in the position or the posture of the second camera is less than a predetermined degree, based on the image of the at least one first camera and the image of the second camera, by:
specifying a position of the workpiece or the component,
specifying the skeletal posture of the worker, or
specifying states of the worker;
or
control the robot, when the change in the position or the posture of the second camera is larger than the predetermined degree, based on the image of the at least one camera, by:
specifying the position of the workpiece or the component,
specifying the skeletal posture of the worker, or
specifying states of the worker.

### Claim 3

A control device, comprising:
a communication interface between a first camera, a second camera, and a robot;
a memory; and
a processor configured to execute a program stored in the memory:
control the robot based on an image of the first camera and an image of the second camera by:
specifying a position of a workpiece or a component,
specifying a skeletal posture of a worker, or
specifying states of the worker;
or
control the robot, when an entirety of the work target appears in an image from the first camera, based on an image from the first camera, by:
specifying a position of the workpiece or the component,
specifying the skeletal posture of the worker, or
specifying states of the worker.