Efficient and robust line matching approach

A method for matching line features for image-based visual servoing (IBVS), said method comprising:

• providing a target image of a workpiece in a desired final position from a prior demonstration, and a current image of the workpiece being manipulated by a robot with a gripper;
• identifying a plurality of lines corresponding with geometric features of the workpiece in each of the target image and the current image;
• determining an initial pairing of the lines in the target image with the lines in the current image based on angle similarity and distance;
• performing an optimization computation using three or more randomly selected pairs of the lines, on a computer having a processor and memory, to determine a rotational and translational transpose which minimize a cost function defined in terms of a direction and a distance between a first and second line in each pair projected into two-dimensional (2D) camera image space;
• computing an error in direction and distance for all line pairs from the initial pairing using the rotational and translational transpose;
• when the error exceeds a threshold, reperforming the optimization computation using a new three or more randomly selected pairs of the lines;
• when the error is less than a threshold, using the rotational and translational transpose to determine an updated pairing of the lines in the target image with the lines in the current image;
• using a best-matched set of line pairs from the updated pairing to compute an IBVS error signal;
• using the error signal to compute robot motion commands;
• and
• controlling the robot based on the robot motion commands.

A system for performing image-based visual servoing robotic manipulation of a workpiece, said system comprising:

• a three-dimensional (3D) camera;
• a robot with a gripper;
• and
• a robot controller having a processor and memory, said controller being in communication with the robot and the camera, being configured for;
• receiving from the camera a target image of the workpiece in a desired final position from a prior demonstration, and a current image of the workpiece being manipulated by the robot with the gripper;
• determining an initial pairing of lines in the target image with lines in a current image based on angle similarity and distance;
• performing an optimization computation using three or more randomly selected pairs of the lines to determine a rotational and translational transpose which minimize a cost function defined in terms of a direction and a distance between a first and second line in each pair projected into two-dimensional (2D) camera image space;
• repeating the optimization computation using a new three or more randomly selected pairs of the lines until an error in direction and distance for all line pairs, computed from the initial pairing using the rotational and translational transpose, is less than a threshold;
• using the rotational and translational transpose to determine an updated pairing of the lines in the target image with the lines in the current image;
• using a best-matched set of line pairs from the updated pairing in an IBVS robot motion control computation;
• and
• controlling the robot to move the workpiece based on the IBVS robot motion control computation.