# US 2018/0222049 A1 · Method of controlling robot, method of teaching robot, and robot system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2018/0222049 A1 - Document ID: US20180222049A1 - Status: Published application - Kind: A1 - Application number: 15/883,711 - Filed: 2018-01-30 - Published: 2018-08-09 - Earliest claimed: 2017-02-09 - Assignees: Canon Kabushiki Kaisha - Applicants: Canon Kabushiki Kaisha - Inventors: HIDEAKI SUZUKI; Keita Dan; Naoki Tsukabe - Web page: https://patentgrep.com/patent/US20180222049A1 ## CPC - B25J 9/1612: {characterised by the hand, wrist, grip control} - B25J 9/161: {Hardware, e.g. neural networks, fuzzy logic, interfaces, processor} - B25J 9/1692: {Calibration of manipulator} - B25J 9/1697: {Vision controlled systems} - G05B 2219/37097: Marker on workpiece to detect reference position - G05B 2219/37567: 3-D vision, stereo vision, with two cameras - G05B 2219/39029: Verify if calibration position is a correct, by comparing with range in rom - G05B 2219/39045: Camera on end effector detects reference pattern - G05B 2219/39397: Map image error directly to robot movement, position with relation to world, base not needed, image based visual servoing - G05B 2219/40613: Camera, laser scanner on end effector, hand eye manipulator, local - Y10S 901/09: Closed loop, sensor feedback controls arm movement - Y10S 901/31: Gripping jaw - Y10S 901/47: Optical - B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50}) - G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G) - Y10S: TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS ## Abstract A robot system includes a robot, a vision sensor, a controller, and an input unit. The vision sensor configured to measure a feature point and obtain a measured coordinate value. The controller configured to control the robot. The input unit configured to receive an input from a user toward the controller. The controller obtains, via the input unit, setting information data on a determination point which is different from the feature point. The robot system uses a coordinate value of the determination point and the measured coordinate value, and determines whether the robot is taking a target position and orientation. ## Claims ### Claim 1 A method of controlling a robot comprising: measuring a measured coordinate value of a feature point by using a vision sensor supported by the robot; computing a converted amount of positional difference, the converted amount of positional difference being obtained by converting an amount of positional difference between the measured coordinate value and a predetermined target coordinate value of the feature point, to an amount of positional difference of a determination point different from the feature point; and comparing the converted amount of positional difference with a first threshold, and determining whether a posture of the robot is within a target range. ### Claim 2 (depends on 1) The method according to claim 1, wherein the robot has a robot hand, wherein the determination point is a freely-selected point within a region, and wherein the region is obtained by adding a holding region in which the robot hand is able to hold a workpiece, to an outer shape of the workpiece held by the robot hand. ### Claim 3 (depends on 1) The method according to claim 1, wherein the determination point is an origin of a coordinate system of the vision sensor, and wherein the computing comprises: determining a coordinate transformation matrix including a rotation matrix and a translation vector which transform the measured coordinate value to the target coordinate value, and determining a length of the translation vector, as the converted amount of positional difference. ### Claim 4 (depends on 1) The method according to claim 1, wherein the determination point is a point other than an origin of a coordinate system of the vision sensor, and wherein the computing comprises: determining a coordinate transformation matrix including a rotation matrix and a translation vector which transform the measured coordinate value to the target coordinate value, transforming a coordinate value of the determination point by multiplying the rotation matrix by the coordinate value of the determination point and adding a resultant value and the translation vector, and determining a distance between the coordinate value of the determination point obtained before the coordinate transformation and a coordinate value of the determination point obtained after the coordinate transformation, as the converted amount of positional difference. ### Claim 5 (depends on 1) The method according to claim 1, comprising repeating the measuring, the computing, the comparing, and a correcting motion based on the amount of positional difference until both the positional difference and the converted amount of positional difference are equal to or smaller than the first threshold. ### Claim 6 (depends on 1) The method according to claim 1, further comprising causing the robot to perform predetermined work if both the converted amount of positional difference and an amount of positional difference between the measured coordinate value and the target coordinate value are equal to or smaller than the first threshold, and if an amount of positional difference between the measured coordinate value and the target coordinate value are equal to or smaller than a second threshold. ### Claim 7 (depends on 1) The method according to claim 1, wherein the vision sensor is a stereo camera having a first camera and a second camera. ### Claim 8 (depends on 1) The method according to claim 1, wherein the robot includes a robot hand, and wherein the vision sensor is fixed to the robot hand. ### Claim 9 (depends on 1) The method according to claim 1, wherein the robot includes a robot hand, and wherein the vision sensor is held by the robot hand in a state where the vision sensor is positioned by a positioning mechanism. ### Claim 10 A method of teaching a robot, comprising: measuring a measured coordinate value of a feature point through a measurement using a vision sensor supported by the robot; computing a converted amount of positional difference, the converted amount of positional difference being obtained by converting an amount of positional difference between the measured coordinate value and a predetermined target coordinate value of the feature point, to an amount of positional difference of a determination point different from the feature point, comparing at least the converted amount of positional difference with a first threshold, and correcting a posture of the robot depending on the amount of positional difference if the amount of positional difference exceeds the first threshold, and repeating the measuring, the computing, and the correcting, and setting, as a teach point, a posture of the robot obtained in a state where the converted amount of positional difference is equal to or smaller than the first threshold. ### Claim 11 (depends on 10) The method according to claim 10, wherein the robot includes a robot hand, and wherein the vision sensor is held by the robot hand in a state where the vision sensor is positioned by a positioning mechanism. ### Claim 12 (depends on 10) The method according to claim 10, wherein the vision sensor is a stereo camera having a first camera and a second camera. ### Claim 13 A robot system comprising: a robot; a vision sensor supported by the robot; and a controller configured to control the robot and the vision sensor, wherein the controller is configured to calculate a transformation parameter which indicates an amount of positional difference between a measured coordinate value of a feature point obtained by using the vision sensor and a predetermined target coordinate value of the feature point, and correct a posture of the robot so that a converted amount of positional difference is equal to or smaller than a predetermined threshold, wherein the converted amount of positional difference is calculated by applying the transformation parameter to coordinates of a determination point positioned at a position different from the feature point. ### Claim 14 A robot system comprising: a robot; a vision sensor configured to measure a feature point and obtain a measured coordinate value; a controller configured to control the robot; and an input unit configured to receive an input from a user toward the controller, wherein the controller obtains, via the input unit, setting information data on a determination point which is different from the feature point, and wherein the robot system uses a coordinate value of the determination point and the measured coordinate value, and determines whether the robot is taking a target position and orientation. ### Claim 15 (depends on 14) The robot system according to claim 14, further comprising a display configured to display a setting window used to set a coordinate value of the determination point, wherein the display displays a plurality of alternatives of the determination point on the setting window, and allows a user to select one of the plurality of alternatives of the determination point. ### Claim 16 (depends on 15) The robot system according to claim 15, wherein the display includes a display area used to display a three-dimensional image, and wherein the display displays a relative relationship between the feature point and the determination point in the display area. ### Claim 17 (depends on 16) The robot system according to claim 16, wherein the display area superimposes the feature point and the determination point on at least one of the robot and the vision sensor. ### Claim 18 (depends on 14) The robot system according to claim 14, wherein the vision sensor is a stereo camera having a first camera and a second camera.