# US 11,110,599 B2 · Robot, control device of robot, and method of teaching position of robot > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,110,599 B2 - Document ID: US11110599B2 - Status: Granted - Kind: B2 - Application number: 16/337,584 - Filed: 2017-09-26 - Published: 2020-01-30 - Granted: 2021-09-07 - Earliest claimed: 2016-09-28 - Assignees: Kawasaki Jukogyo Kabushiki Kaisha - Applicants: Kawasaki Jukogyo Kabushiki Kaisha - Inventors: Hirohiko Goto; Tetsuya Yoshida; Haruhiko Tan; Kazuo Fujimori; Katsuhiro Yamashita; Masahiko Sumitomo - Web page: https://patentgrep.com/patent/US11110599B2 ## CPC - B25J 9/1612: {characterised by the hand, wrist, grip control} - G05B 19/42: Recording and playback systems, i.e. in which the program is recorded from a cycle of operations, e.g. the cycle of operations being manually controlled, after which this record is played back on the same machine - B25J 9/1697: {Vision controlled systems} - B25J 19/023: {including video camera means} - H10P 72/0606: {Position monitoring, e.g. misposition detection or presence detection} - H10P 72/3402: {Mechanical parts of transfer devices} - H10P 72/3411: {involving loading and unloading of wafers} - H10P 72/53: {using optical controlling means} - H10P 72/7602: {the wafers being placed on a robot blade or gripped by a gripper for conveyance} - G05B 2219/45031: Manufacturing semiconductor wafers - B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50}) - G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G) - H10P: GENERIC PROCESSES OR APPARATUS FOR THE MANUFACTURE OR TREATMENT OF DEVICES COVERED BY CLASS H10 ## Abstract A control device of a robot includes: an image data acquirer acquiring image data taken by a camera, the image data including a hand teaching substrate and substrate placing portion, the teaching substrate arranged as a teaching target at a substrate target position; a virtual substrate information generator generating information of a virtual substrate virtually arranged at the hand substrate placing portion in the image data; an operating unit receiving an input and generate operation information corresponding to the input; a screen display unit displaying an image of the teaching substrate and an image of the virtual substrate on a screen; a robot operation control unit controlling an operation of the robot arm in accordance with the operation information; and a teaching data record unit storing, as teaching data, a position of the hand when the virtual substrate has coincided with the teaching substrate. ## Claims ### Claim 1 A robot configured to carry a substrate to a substrate target position at which the substrate is to be placed, the robot comprising: a robot arm; a hand attached to a tip end of the robot arm; a camera fixed and attached to a portion of the hand other than a substrate placing portion of the hand so as to take an image of the substrate placing portion, the substrate being placed at the substrate placing portion; an image data acquirer configured to acquire image data taken by the camera, the image data including a teaching substrate and the substrate placing portion of the hand, the teaching substrate being arranged as a teaching target at the substrate target position; a virtual substrate information generator configured to generate information of a virtual substrate virtually arranged at the substrate placing portion of the hand in the image data; an operating unit configured to receive an input and generate operation information corresponding to the input; a screen display unit configured to display an image of the teaching substrate and an image of the virtual substrate on a screen; a robot operation control unit configured to control an operation of the robot arm in accordance with the operation information generated by the operating unit; and a teaching data record unit configured to store, as teaching data, a position of the hand when the virtual substrate has coincided with the teaching substrate. ### Claim 2 (depends on 1) The robot according to claim 1, wherein the camera is fixed and attached to a base end portion of the hand. ### Claim 3 (depends on 1) The robot according to claim 1, further comprising a distance information calculator configured to calculate distance information from the substrate placing portion to the teaching substrate based on the image data of the camera, wherein the screen display unit displays on the screen an operation guide configured to guide a position of the hand based on the distance information from the substrate placing portion to the teaching substrate such that the virtual substrate coincides with the teaching substrate. ### Claim 4 (depends on 3) The robot according to claim 3, wherein the distance information calculator calculates the distance information from the substrate placing portion to the teaching substrate by pattern matching between the image of the virtual substrate and the image of the teaching substrate. ### Claim 5 (depends on 3) The robot according to claim 3, wherein the operation guide includes: an upward-downward guide portion configured to guide the position of the hand in an upper-lower direction; a leftward-rightward guide portion configured to guide the position of the hand in a left-right direction; and a forward-backward guide portion configured to guide the position of the hand in a front-rear direction. ### Claim 6 (depends on 5) The robot according to claim 5, wherein the screen display unit includes a touch panel configured such that an operator touches the upper-downward guide portion, the left-right direction guide portion, or the front-rear direction guide portion on the screen to input the position of the hand in the upper-lower direction, the left-right direction, or the front-rear direction. ### Claim 7 (depends on 3) The robot according to claim 3, wherein: when the hand is directed in such a direction that the virtual substrate coincides with the teaching substrate, the operation guide is displayed in a first color; and when the hand is not directed in the direction, the operation guide is displayed in the other color. ### Claim 8 (depends on 1) The robot according to claim 1, wherein the operating unit and the display unit are mounted on a portable operation terminal. ### Claim 9 A control device of a robot, the robot comprising: a robot arm; a hand attached to a tip end of the robot arm; and a camera fixed and attached to a portion of the hand other than a substrate placing portion of the hand so as to take an image of the substrate placing portion, a substrate being placed at the substrate placing portion, wherein the robot carries the substrate to a substrate target position at which the substrate is to be placed, the control device comprising: an image data acquirer configured to acquire image data taken by the camera, the image data including a teaching substrate and the substrate placing portion of the hand, the teaching substrate being arranged as a teaching target at the substrate target position; a virtual substrate information generator configured to generate information of a virtual substrate virtually arranged at the substrate placing portion of the hand in the image data; an operating unit configured to receive an input and generate operation information corresponding to the input; a screen display unit configured to display an image of the teaching substrate and an image of the virtual substrate on a screen; a robot operation control unit configured to control an operation of the robot arm in accordance with the operation information; and a teaching data record unit configured to store, as teaching data, a position of the hand when the virtual substrate has coincided with the teaching substrate. ### Claim 10 A method of teaching a position of a hand of a robot configured to carry a substrate to a substrate target position at which the substrate is to be placed, the method comprising: arranging a teaching substrate as a teaching target at the substrate target position; fixing and attaching a camera to a portion of the hand other than a substrate placing portion of the hand such that the camera takes an image of the substrate placing portion, the substrate being placed at the substrate placing portion; moving the robot to a predetermined position at which the camera is able to taken an image of the teaching substrate and the substrate placing portion of the hand; taking the image of the teaching substrate and the substrate placing portion of the hand by the camera; generating information of a virtual substrate virtually arranged at the substrate placing portion of the hand in image data taken by the camera; receiving an input and generating operation information corresponding to the input by an operating unit; displaying an image of the teaching substrate and an image of the virtual substrate on a screen by a screen display unit; controlling an operation of the robot arm in accordance with the operation information; and storing in a storage unit as teaching data, a position of the hand when the virtual substrate has coincided with the teaching substrate. ### Claim 11 (depends on 4) The robot according to claim 4, wherein the operation guide includes: an upward-downward guide portion configured to guide the position of the hand in an upper-lower direction; a leftward-rightward guide portion configured to guide the position of the hand in a left-right direction; and a forward-backward guide portion configured to guide the position of the hand in a front-rear direction. ### Claim 12 (depends on 4) The robot according to claim 4, wherein: when the hand is directed in such a direction that the virtual substrate coincides with the teaching substrate, the operation guide is displayed in a first color; and when the hand is not directed in the direction, the operation guide is displayed in the other color. ### Claim 13 (depends on 5) The robot according to claim 5, wherein: when the hand is directed in such a direction that the virtual substrate coincides with the teaching substrate, the operation guide is displayed in a first color; and when the hand is not directed in the direction, the operation guide is displayed in the other color. ### Claim 14 (depends on 6) The robot according to claim 6, wherein: when the hand is directed in such a direction that the virtual substrate coincides with the teaching substrate, the operation guide is displayed in a first color; and when the hand is not directed in the direction, the operation guide is displayed in the other color. ### Claim 15 (depends on 2) The robot according to claim 2, wherein the operating unit and the display unit are mounted on a portable operation terminal. ### Claim 16 (depends on 3) The robot according to claim 3, wherein the operating unit and the display unit are mounted on a portable operation terminal. ### Claim 17 (depends on 4) The robot according to claim 4, wherein the operating unit and the display unit are mounted on a portable operation terminal. ### Claim 18 (depends on 5) The robot according to claim 5, wherein the operating unit and the display unit are mounted on a portable operation terminal. ### Claim 19 (depends on 6) The robot according to claim 6, wherein the operating unit and the display unit are mounted on a portable operation terminal. ### Claim 20 (depends on 7) The robot according to claim 7, wherein the operating unit and the display unit are mounted on a portable operation terminal.