Robot controlling method for tracking a moving object using a visual sensor

A method of controlling a robot system for performing an operation on an object being conveyed along a conveying path by a conveyor, said robot system includes conveyance amount detecting means for detecting an amount of conveyance of the object by said conveyor and outputting a detection signal representing the conveyance amount, object detecting means disposed in a vicinity of the conveying path for detecting the object being conveyed by said conveyor, visual sensor means for visually detecting a position and posture of the object being conveyed by the conveyor and having a camera disposed in a vicinity of the conveying path downstream of said object detecting means, and a robot controller, said method comprising the steps of:

• (a) setting a fixed base coordinate system to a robot;
• (b) setting a tracking coordinate system to move in synchronism with the conveying path of said conveyor;
• (c) placing a reference object on said conveyor and conveying the reference object by the conveyor;
• (d) obtaining an image of said reference object by said camera at a position where the reference object is conveyed a first predetermined distance from a position where the reference object is detected by said object detecting means;
• (e) analyzing said image obtained in said step (d) by said visual sensor means to detect a position and posture of the reference object;
• (f) teaching the robot at least one point on the reference object at a position where the reference object is conveyed a second predetermined distance from the position where the reference object is detected by said object detection means;
• (g) placing an object of operation having the same dimension as that of the reference object on said conveyor and conveying the object by said conveyor;
• (h) obtaining an image of the object of operation by said camera at an instant when said visual sensor recognizes that the object of operation has been conveyed by said first predetermined distance based on the detection signal of said conveyance amount detecting means from an instant when said visual sensor recognizes that the object is detected by said object detecting means;
• (i) analyzing said image obtained in said step (h) by the visual sensor means to detect a position and posture of said object;
• (j) comparing the position/posture of said reference object detected in said step (e) and the position/posture of said object detected in said step (i) and obtaining a deviation therebetween;
• (k) starting a tracking movement of the robot at an instant when said robot controller recognizes that the object has been conveyed by a third predetermined distance based on the detection signal from said conveyance amount detecting means from the instant when the robot controller recognizes that the object of operation is detected by said object detecting means;
• (l) obtaining a position of a target point in the tracking coordinate system based on the position of the point taught in said step (f), an output signal of said object detecting means and the detection signal of said conveyance amount detecting means;
• (m) converting the position of the target point in said tracking coordinate system into a position of the target point in said base coordinate system;
• (n) correcting the position of the target point converted in said step (m) based on the deviation obtained in said step (j);
• and
• (o) controlling the robot based on the position of the target point corrected in said step (n).