Optical axis calibration of robotic camera system

A robotic camera system comprising:

• a serial robot having an end-effector;
• an ophthalmic microscope that is (i) integral with a digital camera having a focus motor and (ii) connectable to the end-effector of the serial robot, wherein the end-effector and the digital camera move within a robot motion coordinate frame;
• and
• an electronic control unit (ECU) in communication with the digital camera and configured to:
• acquire, using the digital camera, a reference image of a pixelated target graphic on an image plane having an optical coordinate frame, wherein the digital camera has a variable optical working distance between the digital camera and the pixelated target graphic that is controlled by a focus motor;
• receive input signals, including (i) a depth measurement from a depth sensor, the depth measurement being indicative of a linear distance between the digital camera and the pixelated target graphic, and (ii) a set of joint position signals collectively describing a position of the digital camera in the robot motion coordinate frame, the robot motion coordinate frame having an x-axis, a y-axis, and a z-axis;
• determine a roll offset and a pitch offset of a target point within the reference image with respect to the robot motion coordinate frame while moving the serial robot through a first calibrated motion sequence;
• determine, after determining the roll offset and the pitch offset, each of an x-axis offset, a y-axis offset, and a z-axis offset of the target point with respect to the robot motion coordinate frame while moving the serial robot through a second calibrated motion sequence, including extracting the z-axis offset from a lookup table indexed by the variable optical working distance and one or more parameters of the focus motor;
• store the roll offset, the pitch offset, the x-axis offset, the y-axis offset, and the z-axis offset in a transformation matrix within memory of the ECU;
• and
• control a third motion sequence of the serial robot using the transformation matrix during a subsequent operation of the robotic camera system, wherein the subsequent operation of the robotic camera system is performed using the ophthalmic microscope as part of an eye surgery of the eye of the human patient.

A non-transitory computer-readable medium on which is recorded instructions, execution of which by a processor causes the processor, when used with a robot camera system having a digital camera connected to an end-effector of a serial robot, to:

• acquire, from a digital camera that is integral with an ophthalmic microscope, a reference image of a pixelated target object on an image plane having an optical coordinate frame;
• receive input signals, including a depth measurement from a laser distance meter or an optical sensor, the depth measurement being indicative of a linear distance between the digital camera and the pixelated target object, and a set of joint position signals collectively describing a position of the digital camera in a robot motion coordinate frame, the robot motion coordinate frame having an x-axis, a y-axis, and a z-axis;
• determine a roll offset and a pitch offset of a target point within the reference image with respect to the robot motion coordinate frame while moving the serial robot through a first calibrated motion sequence;
• determine, after determining the roll offset and the pitch offset, each of an x-axis offset, a y-axis offset, and a z-axis offset of the target point with respect to the robot motion coordinate frame while moving the serial robot through a second calibrated motion sequence;
• store the roll offset, the pitch offset, the x-axis offset, the y-axis offset, and the z-axis offset in a transformation matrix within the computer-readable medium, thereby calibrating a robotic camera system having the digital camera;
• and
• control a third motion sequence of the serial robot during a subsequent operation of the robotic camera system using the transformation matrix, wherein the subsequent operation of the robotic camera system is performed using the ophthalmic microscope as part of an eye surgery of an eye of a human patient.