Systems and methods for changing the direction of view during video guided clinical procedures using real-time image processing

A system for rendering a target image of a virtual camera, the system comprising:

• a non-transitory computer-readable medium storing instructions;
• and
• one or more processors configured to execute the instructions, which cause the one or more processors to:
• capture a source image by a source camera comprising camera-head and a rigid endoscope that are real;
• define a rotation between the source camera and the virtual camera for the target image;
• derive a virtual camera model based on a virtual focal length for the target image;
• and
• generate the target image of the virtual camera and at the virtual focal length by mapping a plurality of pixels in the target image into a pixel in the source image using a mapping function, the mapping function based on a source camera model, the rotation, and an inverse of the virtual camera model.

A computer-implemented method for rendering a target image of a virtual camera, the method comprising:

• finding, in a source image, a location of an optical axis, the source image captured by a source camera at a capture azimuth, and a comprising a camera-head and a rigid endoscope that are real;
• updating a camera model of the source camera, the source camera has, at a first azimuth, a focal length, a radial distortion, and an optical axis, and wherein updating the camera model comprises updating the camera model for the capture azimuth angularly displaced from the first azimuth;
• defining a line normal to a plane that contains, or passes close to, a mechanical axis and an optical axis of the source camera;
• defining a rotational relationship between the virtual camera and the source camera as a rotation about the line normal to the plane;
• computing a virtual focal length and a location of a virtual principal point for the virtual camera, and deriving a camera model of the virtual camera based on the virtual focal length and the location of the virtual principal point;
• and
• generating the target image by mapping a plurality of pixels of the target image into a point in the source image, the mapping with a mapping function comprising the camera model of the source camera, the rotational relationship, and an inverse of the camera model of the virtual camera.

A system for rendering a target image of a virtual camera, the system comprising:

• a non-transitory computer-readable medium storing instructions;
• and
• one or more processors configured to execute the instructions, which cause the one or more processors to:
• find, in a source image, a location of an optical axis, the source image captured by source camera at a capture azimuth, and the source camera comprising a camera-head and a rigid endoscope that are real;
• update a camera model of the source camera, the source camera has, at a first azimuth, a focal length, a radial distortion, and an optical axis, and wherein updating the camera model comprises updating the camera model for the capture azimuth angularly displaced from the first azimuth;
• define a line normal to a plane that contains, or passes close to, a mechanical axis and an optical axis of the source camera;
• define a rotational relationship between the virtual camera and the source camera as a rotation about the line normal to the plane;
• compute a virtual focal length and a location of a virtual principal point for the virtual camera, and derive a camera model of the virtual camera based on the virtual focal length and the location of the virtual principal point;
• and
• generate the target image by mapping a plurality of pixels of the target image into a point in the source image, the mapping with a mapping function comprising the camera model of the source camera, the rotational relationship, and an inverse of the camera model of the virtual camera.