Method and system for performing automatic camera calibration for robot control

A method for performing robot control, the method comprising:

• generating, by a robot control system, a movement command for outputting to a communication interface of the robot control system, wherein the communication interface is configured to communicate with a robot having a robot arm with a calibration pattern disposed thereon, and is configured to communicate with a camera having a camera field of view;
• outputting, by the robot control system, the movement command to the communication interface, wherein the movement command is for controlling the robot arm to move the calibration pattern to at least one location in the camera field of view;
• receiving, by the robot control system, a calibration image from the communication interface, wherein the communication interface is configured to receive the calibration image from the camera, the calibration image being an image of the calibration pattern at the at least one location in the camera field of view, and wherein the calibration pattern includes a plurality of pattern elements having respective defined pattern element locations in a pattern coordinate system, wherein the pattern coordinate system is a coordinate system defined with respect to a location and orientation of the calibration pattern;
• determining, by the robot control system, a plurality of image pattern element locations from the calibration image, wherein the plurality of image pattern element locations indicate respective locations at which the plurality of pattern elements appear in the calibration image;
• determining, by the robot control system, a first estimate of a first intrinsic camera parameter based on the plurality of image pattern element locations and based on the defined pattern element locations;
• determining, by the robot control system, after the first estimate of the first intrinsic camera parameter has been determined, an estimate of a second intrinsic camera parameter based on the first estimate of the first intrinsic camera parameter and based on the plurality of image pattern element locations;
• determining, by the robot control system, a second estimate of the first intrinsic camera parameter based on the estimate of the second intrinsic camera parameter, and based on the plurality of image pattern element locations and the defined pattern element locations;
• receiving, by the robot control system, a subsequent image from the communication interface, wherein the communication interface is configured to receive the subsequent image from the camera after the second estimate of the first intrinsic camera parameter and the estimate of the second intrinsic camera parameter have been determined;
• generating, by the robot control system, a subsequent movement command based on the subsequent image and based on the second estimate of the first intrinsic camera parameter and the estimate of the second intrinsic camera parameter;
• and
• outputting, by the robot control system, the subsequent movement command to the communication interface, wherein the subsequent movement command is for controlling placement of the robot arm.

A non-transitory computer-readable medium having instructions stored thereon that, when executed by a control circuit of a robot control system, causes the control circuit to perform camera calibration by:

• generating a movement command for outputting to a communication interface of the robot control system, wherein the communication interface is configured to communicate with a robot having a robot arm with a calibration pattern disposed thereon, and with a camera having a camera field of view;
• outputting the movement command to the communication interface, wherein the movement command is for controlling the robot arm to move the calibration pattern to at least one location in the camera field of view;
• receiving a calibration image from the communication interface, wherein the communication interface is configured to receive the calibration image from the camera, the calibration image being an image of the calibration pattern at the at least one location in the camera field of view, and wherein the calibration pattern includes a plurality of pattern elements having respective defined pattern element locations in a pattern coordinate system, wherein the pattern coordinate system is a coordinate system defined with respect to a location and orientation of the calibration pattern;
• determining a plurality of image pattern element locations from the calibration image, wherein the plurality of image pattern element locations indicate respective locations at which the plurality of pattern elements appear in the calibration image;
• determining a first estimate of a first intrinsic camera parameter based on the plurality of image pattern element locations and based on the defined pattern element locations;
• determining, after the first estimate of the first intrinsic camera parameter has been determined, an estimate of a second intrinsic camera parameter based on the first estimate of the first intrinsic camera parameter and based on the plurality of image pattern element locations;
• determining a second estimate of the first intrinsic camera parameter based on the estimate of the second intrinsic camera parameter, and based on the plurality of image pattern element locations and the defined pattern element locations
• wherein the instructions, when executed by the control circuit, further cause the control circuit, after the camera calibration is performed, to receive a subsequent image from the communication interface, wherein the communication interface is configured to receive the subsequent image from the camera, and wherein the instructions, when executed by the control circuit, further cause the control circuit to generate a subsequent movement command for controlling placement of the robot arm of the robot, wherein the subsequent movement command is generated based on the subsequent image and based on the second estimate of the first intrinsic camera parameter and the estimate of the second intrinsic camera parameter.