Automatic end effector tip position initialization mechanism for micromanipulation system

An automatic end effector tip position initialization method for a micromanipulation system, wherein the automatic end effector tip position initialization method is realized by an automatic end effector tip position initialization device for the micromanipulation system, the automatic end effector tip position initialization device comprises two sets of micromanipulators, a macro-micro integrated imaging system, and a computer/user interface;

• the two sets of micromanipulators are three-axis Cartesian manipulators, the three-axis Cartesian manipulators are symmetrically distributed on left and right sides of a microscope, the three-axis Cartesian manipulators are configured to control an end effector and a petri dish respectively, and the three-axis Cartesian manipulators are defined as a left arm and a right arm respectively;
• the macro-micro integrated imaging system comprises a micro-image part and a macro-image part, the micro-image part is acquired by a 20-fold objective lens digital camera, and the macro-image part is acquired by a macro camera, and macro-micro images will eventually be transmitted to the computer/user interface respectively;
• the automatic end effector tip position initialization method comprises the following steps:
• S1, positioning an end effector tip at a macro level by acquiring a macro image of a micromanipulation space by the macro camera, moving the end effector to analyze a position of the end effector tip in an image coordinate system, calculating a camera pose by using a black-and-white square-based fiducial marker—ArUco Tag for positioning, and realizing a three-dimensional positioning of the end effector tip by using a triangulation method, and moving the end effector to an area near a center of a focal plane based on visual servoing;
• S2, positioning the end effector tip at a micro level by conducting the positioning in a 4×2×2 mm rectangular region, using a shadow projected by the end effector in a microscope image to position a shaft and the tip of the end effector, and moving the end effector tip to the focal plane;
• S3, acquiring calibration data based on the visual servo, wherein the calibration data is calibration data containing micro-image position information and micromanipulator encoder position information, calibrating the end effector tip and the petri dish, and obtaining a left calibration matrix and a right calibration matrix reflecting a transformation relationship between left and right micromanipulator coordinates and image coordinates;
• and
• S4, generating an intuitive human-computer interaction interface based on a mouse and a keyboard according to position information acquired by S1-S3, wherein microscopic information in the intuitive human-computer interaction interface is configured for a local precision micromanipulation, and macroscopic information is configured for helping an operator to master a global environment.