Image-based guidance for robotic wire pickup

An automated system comprising:

• a robot arm comprising:
• links and joints coupled to form a kinematic chain;
• and
• a plurality of robot arm motors configured to drive movement of a distal end of the robot arm;
• a tool head coupled to the distal end of the robot arm and comprising a wire gripper motor;
• a wire gripper movably coupled to the tool head and operatively coupled to the wire gripper motor, wherein the wire gripper comprises a pair of gripper fingers which are configured for synchronized movements in mutually opposite directions for opening or closing the wire gripper, and wherein the gripper fingers have respective tips which form a tip of the wire gripper;
• a wire holder configured for clamping a wire;
• camera means mounted to the distal end of the robot arm and having first and second fields of view which intersect in a volume of space that includes the tip of the wire gripper and the wire holder;
• and
• a computer system configured to control operation of the robot arm motors and wire gripper motor, wherein the computer system is configured to perform steps comprising:
• (a) activating the camera means to take camera images from different viewpoints while a portion of a wire is being held by the wire holder, wherein the camera images include (1) an image of a portion of the wire not held by the wire holder and disposed on an opposite side of the wire holder and (2) an image of a portion of a tip of the wire gripper;
• (b) processing the camera images to generate (1) an estimate of a deviation of an orientation of the wire gripper from being parallel with the portion of the wire not held by the wire holder and (2) an estimate of a distance separating the portion of the wire not held by the wire holder from a center of the tip of the wire gripper;
• (c) controlling the robot arm motors to align the wire gripper with the portion of the wire not held by the wire holder based on the estimates generated in step (b);
• and
• (d) controlling the wire gripper motor to cause the wire gripper to grip the wire while aligned with the portion of the wire not held by the wire holder.

A method for guiding a robot having a wire gripper and at least one camera, the method comprising:

• (a) taking camera images from different viewpoints while the wire gripper is stationary;
• (b) determining an orientation of a first portion of a wire relative to the wire gripper based on the camera images taken in step (a);
• (c) controlling the robot to rotate the wire gripper to be parallel to the first portion of the wire based on the orientation relative to the wire gripper determined in step (b);
• (d) taking camera images from different viewpoints while the wire gripper is parallel to the first portion of the wire;
• (e) calculating, based on the camera images taken in step (d), an estimated distance separating a center of a tip of the wire gripper and the first portion of the wire;
• (f) determining that the estimated distance calculated in step (e) is not less than a first threshold;
• (g) controlling the robot to move the wire gripper closer to the first portion of the wire by the estimated distance calculated in step (e);
• (h) taking camera images from different viewpoints after the wire has been moved by the estimated distance calculated in step (e);
• (i) calculating, based on the camera images taken in step (h), an estimated distance separating the center of the tip of the wire gripper and the first portion of the wire;
• (j) determining that the estimated distance calculated in step (i) is less than the first threshold;
• and
• (k) controlling a pair of gripper fingers of the wire gripper to move toward each other in response to step (j).

A method for guiding a robot having a wire gripper comprising a pair of gripper fingers and having at least one camera, the method comprising:

• (a) holding a portion of a wire in a wire holder of a wire holding device, wherein the wire has a wire contact disposed on one side of the wire holder;
• (b) moving the wire gripper to a predetermined location in proximity to the wire holder;
• (c) opening the wire gripper by moving the gripper fingers apart either before or after step (b);
• (d) subsequent to step (c), taking camera images from different viewpoints, wherein the camera images include (1) an image of a portion of the wire not held by the wire holder and disposed on an opposite side of the wire holder and (2) images of respective portions of the tips of the gripper fingers of the wire gripper in an open state;
• (e) deriving, from the camera images taken in step (d) visual feedback data representing an orientation of the wire gripper relative to the portion of the wire not held by the wire holder;
• (f) controlling, based on the visual feedback data derived in step (e), the robot to rotate the wire gripper so that the orientation of the wire gripper matches the orientation of the portion of the wire not held by the wire holder;
• (g) subsequent to step (f), taking camera images from different viewpoints, wherein the camera images include (1) an image of the portion of the wire not held by the wire holder and (2) images of respective portions of the tips of the gripper fingers of the wire gripper in an open state;
• (h) deriving, from the camera images taken in step (g), visual feedback data representing a position of a midpoint between the tips of the gripper fingers relative to the portion of the wire not held by the wire holder;
• (i) controlling, based on the visual feedback data derived in step (h), the robot to move the wire gripper to a position where the gripper fingers are on opposite sides of the portion of the wire not held by the wire holder;
• and
• (j) controlling the wire gripper to cause the gripper fingers to grip the portion of the wire not held by the wire holder.