Autonomous mobile grabbing method for mechanical arm based on visual-haptic fusion under complex illumination condition

An autonomous mobile grabbing method for a mechanical arm based on visual-haptic fusion under a complex illumination condition, wherein under the complex illumination condition, visible light and depth information based on vision and haptic information of an object are fused to complete a process from fuzzy identification of the object to precise identification and grabbing, and the specific steps are as follows:

• step S1:
• under the complex illumination condition, introducing depth information which is less influenced by illumination change, identifying an approximate position of a target object in combination with visible light information, and sending a control instruction;
• step S2:
• after a mobile platform receives the instruction, converting a coordinate system of the object into a coordinate system of the mobile platform, and navigating the mobile platform to a position near the target object;
• step S3:
• starting a depth camera on the mobile platform again to obtain pose information of the target object, converting coordinates of the object into a base coordinate system of the mechanical arm, and controlling the mechanical arm to move to a position near the target object;
• step S4:
• starting a depth camera at an end of the mechanical arm to obtain pose information of the target object, converting coordinates of the object into a base coordinate system of the mechanical arm, and controlling the mechanical arm to move to a target point;
• step S5:
• adjusting the pose of the mechanical arm according to contact force information between a 6-dimensional force sensor at the end of the mechanical arm and an external environment;
• step S6:
• acquiring haptic information by using sensors at fingertips of a mechanical hand, fusing the haptic information and the visual information to select the optimal grabbing pose of the target object, and determining the optimal grabbing force;
• and
• step S7:
• completing the precise grabbing of the target object according to the determined grabbing pose and grabbing force.