System and method for robot teaching based on RGB-D images and teach pendant

A system for robot teaching based on RGB-D images and a teach pendant, comprising:

• a RGB-D camera,
• a host computer,
• a posture teach pendant, and
• an AR teaching system;
• wherein, the RGB-D camera and the posture teach pendant are communicated with the host computer;
• the RGB-D camera is set in a physical working environment, and the AR teaching system comprises an AR registration card located in the physical working environment, an AR module running in the host computer, a virtual robot model comprising a robot end effector, a path planning unit and a posture teaching unit;
• RGB images and depth images of the physical working environment are collected by the RGB-D camera in real time and are sent to the host computer;
• the AR module sets a position of a virtual camera in a virtual scene, and overlays the virtual robot model on the RGB images to complete AR registration;
• the path planning unit displays the RGB images and depth images, and a teaching programmer interactively selects path points of the robot end effector on the RGB images, thereby calculating a 3D coordinate of each of the path points of the robot end effector in a basic coordinate system of the virtual robot model based on a transformation between the depth images and the RGB images;
• the posture teach pendant is operated by the teaching programmer to generate posture data;
• the posture teaching unit receives the posture data of the posture teach pendant in real time when the host computer sequentially reads the 3D coordinates of the path points of the robot end effector in the basic coordinate system of the virtual robot model, and then the virtual robot model moves based on the 3D coordinates;
• during the movement, the received posture data is recorded as the posture data corresponding to a 3D coordinate of a path point where the virtual robot model is located, so that the virtual robot model moves in accordance with teaching postures and positions to complete the robot teaching.