Method Of Calibrating Camera To Robot, System, And Non-Transitory Computer-Readable Storage Medium Storing Computer Program

A system configured to execute calibration processing of a camera to a robot, the system comprising:

• a camera configured to take an image of a robot arm of the robot;
• and
• a calibration processor configured to execute the calibration processing of the camera using the image taken by the camera, wherein
• the calibration processor is configured to execute
• (a) processing of estimating pixel coordinate values of a plurality of characteristic points set in advance in the robot arm from the image of the robot arm taken by the camera using a first machine learning model already learned,
• (b) processing of estimating first coordinate values of the plurality of characteristic points in a 3D camera coordinate system from the pixel coordinate values of the plurality of characteristic points using a second machine learning model already learned,
• (c) processing of calculating second coordinate values of the plurality of characteristic points in a 3D robot coordinate system using an encoder value of the robot arm, and
• (d) processing of executing the processing (a) through (c) with respect to a plurality of postures of the robot arm to estimate calibration parameters including an external parameter of the camera using the first coordinate values and the second coordinate values of the plurality of characteristic points in the plurality of postures.

A non-transitory computer-readable storage medium storing a computer program configured to make a processor execute calibration processing of a camera to a robot, the calibration processing comprising:

• (a) processing of estimating pixel coordinate values of a plurality of characteristic points set in advance in a robot arm from an image of the robot arm taken by the camera using a first machine learning model already learned;
• (b) processing of estimating first coordinate values of the plurality of characteristic points in a 3D camera coordinate system from the pixel coordinate values of the plurality of characteristic points using a second machine learning model already learned;
• (c) processing of calculating second coordinate values of the plurality of characteristic points in a 3D robot coordinate system using an encoder value of the robot arm;
• and
• (d) processing of executing the processing (a) through (c) with respect to a plurality of postures of the robot arm to estimate calibration parameters including an external parameter of the camera using the first coordinate values and the second coordinate values of the plurality of characteristic points in the plurality of postures.