# US 2021/0031374 A1 · Robot device controller for controlling position of robot

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0031374 A1
- Document ID: US20210031374A1
- Status: Published application
- Kind: A1
- Application number: 16/923,485
- Filed: 2020-07-08
- Published: 2021-02-04
- Earliest claimed: 2019-08-01
- Assignees: Fanuc Corporation
- Applicants: Fanuc Corporation
- Inventors: Kentaro Koga; Wataru Miyazaki
- Web page: https://patentgrep.com/patent/US20210031374A1

## CPC

- B25J 9/1697: {Vision controlled systems}
- B25J 9/1664: {characterised by motion, path, trajectory planning}
- B25J 9/1687: {Assembly, peg and hole, palletising, straight line, weaving pattern movement}
- G05B 2219/39391: Visual servoing, track end effector with camera image feedback
- G05B 2219/39393: Camera detects projected image, compare with reference image, position end effector
- G05B 2219/39395: Expectation based visual servoing, use of model
- G05B 2219/39397: Map image error directly to robot movement, position with relation to world, base not needed, image based visual servoing
- G05B 2219/40033: Assembly, microassembly
- G05B 2219/40555: Orientation and distance
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)

## Abstract

A first characteristic portion of a first workpiece and a second characteristic portion of a second workpiece are previously determined. A characteristic amount detection unit detects a first characteristic amount related to the position of the first characteristic portion and a second characteristic amount related to the position of the second characteristic portion in an image captured by a camera. A calculation unit calculates, as a relative position amount, the difference between the first characteristic amount and the second characteristic amount. A command generation unit generates a movement command for operating a robot based on a relative position amount in the image captured by the camera and a relative position amount in a predetermined reference image.

## Claims

### Claim 1

A controller for a robot device in which a robot moves a second member so as to adjust the relative position of the second member with respect to a first member, comprising:
a visual sensor capturing images of the first member and the second member;
an operation control unit transmitting a command for driving the robot to the robot; and
an image processing unit for processing an image captured by the visual sensor, wherein a first characteristic portion for detecting the position of the first member and a second characteristic portion for detecting the position of the second member are previously determined,
the image processing unit includes a characteristic amount detection unit for detecting a first characteristic amount related to the position of the first characteristic portion and a second characteristic amount related to the position of the second characteristic portion in an image captured by the visual sensor,
a calculation unit for calculating the difference between the first characteristic amount and the second characteristic amount as a relative position amount, and
a command generation unit for generating a movement command for operating the robot,
the relative position amount in a reference image including the images of the first characteristic portion and the second characteristic portion when the second member is arranged at a target position with respect to the first member is determined,
the command generation unit generate a movement command for operating the robot so that the second member is arranged at the target position with respect to the first member, based on the relative position amount in an image captured by the visual sensor and the relative position amount in the reference image including the images of the first characteristic portion and the second characteristic portion, and
the operation control unit changes the position of the robot based on the movement command.

### Claim 2 (depends on 1)

The controller according to claim 1, wherein the command generation unit calculates the difference between the relative position amounts, which is the difference between the relative position amount in the image captured by the visual sensor and the relative position amount in the reference image,
the movement direction of the position of the robot with respect to the difference between the relative position amounts is previously determined,
a method for calculating the movement amount of the position of the robot with respect to the difference between the relative position amounts is previously determined, and
the command generation unit sets a movement direction and a movement amount of the position of the robot based on the difference between the relative position amounts, and generates a movement command based on the movement direction and movement amount of the position of the robot.

### Claim 3 (depends on 1)

The controller according to claim 1, wherein
the robot device includes a carrier for conveying the first member, and
a control operation for detecting the first characteristic amount and the second characteristic amount by the characteristic amount detecting unit, a control operation for detecting the relative position amount by the calculation unit, and a control operation for generating a movement command by the command generation unit are repeatedly performed while the carrier conveys the first member.

### Claim 4 (depends on 3)

The controller according to claim 3, wherein the movement direction and movement speed of the first member conveyed by the carrier are previously determined, and
the command generation unit generate a movement command based on the movement direction and movement amount of the position of the robot, in which the position of the robot follows the position of the first member conveyed by the carrier.

### Claim 5 (depends on 1)

The controller according to claim 1, wherein the visual sensor constitutes a first visual sensor,
the controller further comprises a second visual sensor for capturing images of the first member and the second member,
a third characteristic portion for detecting the position of the first member and a fourth characteristic portion for detecting the position of the second member are previously determined,
the characteristic amount detection unit detects a third characteristic amount related to the position of the third characteristic portion and a fourth characteristic amount related to the position of the fourth characteristic portion in the image captured by the second visual sensor,
the calculation unit calculates, as a relative position amount, the difference between the third characteristic amount and the fourth characteristic amount,
a relative position amount in the reference image including the images of the third characteristic portion and the fourth characteristic portion when the second member is arranged at a target position with respect to the first member is determined, and
the command generation unit generates, based on a relative position amount in an image captured by the second visual sensor and a relative position amount in the reference image including images of the third characteristic portion and the fourth characteristic portion, a movement command for operating the robot so that the second member is arranged at a target position with respect to the first member, and generates, based on a movement command generated from an image captured by the first visual sensor and a movement command generated from an image captured by the second visual sensor, a movement command to be transmitted to the operation control unit.

### Claim 6 (depends on 5)

The controller according to claim 5, wherein the first visual sensor is arranged so as to capture an image of a portion arranged on one side of the second member in a predetermined direction,
the second visual sensor is arranged so as to capture an image of a portion arranged on the other side of the second member in the predetermined direction, and
the controller determines whether the orientation of the second member with respect to the first member remains within a predetermined determination range based on the movement command generated from the image captured by the first visual sensor and the movement command generated from the image captured by the second visual sensor.
