Automatic welding system and method for large structural parts based on hybrid robots and 3D vision

An automatic welding method for large structural parts based on hybrid robots and 3D vision based on a welding system, which comprises a hybrid robot system composed of a mobile robot (3) and a Multi-Degree-Of-Freedom (MDOF) robot (2) installed above the mobile robot (3), the welding system installed at a tail end of the MDOF robot (2) and used for welding a target workpiece (1), and a 3D vision system installed at the tail end of the MDOF robot (2) or on the welding system, and used for global calibration and positioning of the hybrid robot system, the welding system and the target workpiece;

• wherein the mobile robot (3) comprises a mobile robot chassis (13);
• a shell (15) being fixedly disposed on the mobile robot chassis (13);
• a rechargeable battery pack (11) and a power supply (12) used for providing energy for the whole system, a controller (10) of the MDOF robot (2), and a welding machine (9) of the welding system being disposed in the shell (15);
• the rechargeable battery pack (11) being connected to an external power supply through a power port (14) on the shell (15);
• the MDOF robot (2) comprises an MDOF robot body (6), and the MDOF robot body (6) and a teach pendant being in a signal connection with the controller (10) of the MDOF robot (2) in the shell (15) through cables;
• the welding system comprises the welding machine (9) located in the shell (15), and a welding gun (7) connected to the welding machine (9) and disposed at the tail end of the MDOF robot body (6);
• the 3D vision system comprises a 3D camera (5), the 3D camera (5) being installed at the tail end of the MDOF robot body (6) or on the welding gun (7), and being connected to an industrial personal computer (8) on the shell (15) through a cable, and the industrial personal computer (8) being connected to the controller (10) of the MDOF robot (2) through a cable;
• the 3D camera (5) has a measurement accuracy not lower than 0.5 mm, and a deep frame rate greater than one frame per second;
• and
• the MDOF robot (2) comprises a robot arm with more than six degrees of freedom, and an arm reach of the robot arm being 0.5 m-2 m;
• wherein the welding method comprises the steps of:
• step (1) establishing a coordinate system of the hybrid robot system formed by the mobile robot (3) and the MDOF robot (2);
• step (2) calibrating a relation between the welding system and the coordinate system of the hybrid robot system and a relation between the 3D vision system and the coordinate system of the hybrid robot system;
• step (3) placing the target workpiece (1) in a working area, and aligning coordinates of the workpiece to a global coordinate system;
• step (4) planning a motion path of the mobile robot (3), and a photographing position and pose of the 3D camera (5) in the 3D vision system;
• step (5) generating motion control and welding programs;
• and
• step (6) performing welding by the mobile robot (3);
• wherein step (2) further comprises the steps of:
• step (2.1) calibrating a coordinate relation between the MDOF robot (2) and the 3D camera (5) to obtain a transformation relation between the coordinate system of the vision system of the camera and a coordinate system of the MDOF robot (2);
• step (2.11) ensuring that the 3D camera (5) is calibrated, and obtaining intrinsic parameters of the camera, wherein the intrinsic parameters comprise:
• focal length, position of a principal point, pixel size, resolution, and distortion parameters;
• step (2.12) calibrating the 3D camera (5) and the MDOF robot (2), defining a homogeneous transformation matrix from the tail end of the MDOF robot (2) to a base of the robot as robotTbase, and similarly, defining a transformation matrix from the 3D camera (5) to a target object as camTobj;
• mounting the 3D camera (5) on the MDOF robot (2), photographing a calibration plate, coordinates of which are known, and recording a position and pose of the MDOF robot (2);
• keeping the calibration plate still, changing the position and pose of the MDOF robot (2) multiple times to photograph the calibration plate, wherein two different times of photographing are expressed as:
• robot1Tbase·cam1Trobot1·objTcam1=robot2Tbase·cam2Trobot2·objTcam2;
• robot1Tbase·cam1Trobot1·objTcam1=robot2Tbase·cam2Trobot2·objTcam2 (1)
• wherein a coordinate relation between the camera and the tail end of the MDOF robot is constant, that is, cam1Trobot1=cam2Trobot2=camTrobot,
• cam1Trobot1=cam2Trobot2=camTrobot (2)
• (robot2T−1base·robot1Tbase)·camTrobot=camTrobot·(objTcam2·objT−1cam1) (3)
• the equation is solved through multiple times of photographing to obtain a coordinate transformation relation camTrobot between the 3D camera (5) and the MDOF robot (2);
• a hand-eye transformation relation camTtool of the 3D camera (5) is:
• camTtool=camTrobot·robotTbase·baseTtool=camTrobot·robotTbase·toolT−1base (4)
• step (2.13) performing closed-loop control to obtain a transformation relation between a coordinate system of the 3D camera (5) and a tool coordinate system of a tail end of the welding gun (7), touching a known point on the calibration plate by the tail end of the welding gun to obtain a position P′ (x,y,z) of the known point in the coordinate system of the MDOF robot (2), photographing the calibration plate by the 3D camera to obtain a position P″ (x,y,z) of the known point in the coordinate system of the 3D camera, substituting an energy equation representing a spatial distance between P′ (x,y,z) and P″ (x,y,z) into formula (4) to obtain an initial value of the hand-eye transformation relation camTtool, and performing closed-loop iteration to solve an optimal hand-eye transformation matrix camTtool;
• wherein, the energy equation is:
• P=|P′1(x,y,z)P″1(x,y,z)|+|P′2(x,y,z)P″2(x,y,z)|+ . . . .
• where, |P′1(x,y,z)P″1(x,y,z)| represents an Euclidean distance from P′1(x,y,z) to P″1(x,y,z), and the subscript represents multiple points;
• step (2.2) calibrating a TCP coordinate system of the robot (2) to obtain a position transformation relation of a sharp end of the welding gun (7) in the coordinate system of the MDOF robot (2);
• wherein a TCP calibration method of the robot is a direct input method, a four-point method or a six-point method, and the four-point method specifically comprises the steps of:
• step (2.21) establishing a new TCP coordinate system of the MDOF robot;
• step (2.22) placing a fixed point in a working space of the MDOF robot (2), wherein the fixed point is generally a conical sharp point;
• step (2.23) controlling the pose of the MDOF robot (2) to make a TCP be overlapped with the fixed point in the working space;
• step (2.24) repeating Step (2.23) three times to enable TCPs to move to the same point by changing the pose of the MDOF robot (2);
• and
• step (2.25) under the condition where coordinates of the four TCPs in a world coordinate system are identical, establishing an equation set, solving the equation set to realize position calibration of the tool coordinate system, such that a pose transformation relation of coordinates of the tail end of the welding gun (7) of the welding system in the coordinate system of the MDOF robot (2) is obtained;
• and
• step (2.3) calibrating the coordinate system of the MDOF robot and a coordinate system of the mobile robot to obtain a matrix for transforming the coordinate system of the MDOF robot to a coordinate system of the mobile robot;
• wherein the welding method further comprises the steps of:
• mounting 3D camera on the MDOF robot (2) to photograph the calibration plate, the coordinates of which are known, and recording a position and pose of the mobile robot (3);
• keeping the calibration plate still, changing the position of the mobile robot (3) multiple times as significantly as possible, and then adjusting the robot to photograph the calibration plate again according to the coordinate transformation relation camTrobot between the 3D camera and the tail end of the MDOF robot, wherein two different times of photographing are expressed as:
• baseTBASE robot1Tbase·camTrobot·objTcam1=baseTBASE robot2Tbase·camTrobot·objTcam2;
• then photographing multiple times to solve the equation so as to obtain a transformation relation base TBASE between a base coordinate system of the mobile robot (3) and the basic coordinate system of the system.