# US 12,183,038 B2 · Camera calibration using fiducial markers on surgical tools > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,183,038 B2 - Document ID: US12183038B2 - Status: Granted - Kind: B2 - Application number: 17/495,784 - Filed: 2021-10-06 - Published: 2022-04-07 - Granted: 2024-12-31 - Earliest claimed: 2020-10-06 - Assignees: Asensus Surgical Europe S.à.R.L. - Applicants: Asensus Surgical Europe Sàrl - Inventors: Tal Nir; Lior Alpert - Web page: https://patentgrep.com/patent/US12183038B2 ## CPC - A61B 1/00057: {provided with means for testing or calibration} - G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration - A61B 1/3132: {for laparoscopy} - A61B 90/39: {Markers, e.g. radio-opaque or breast lesions markers} - A61B 2090/363: {Use of fiducial points} - A61B 2090/3937: {Visible markers} - A61B 2090/3983: {Reference marker arrangements for use with image guided surgery} - G06T 2207/10016: Video; Image sequence - G06T 2207/10068: Endoscopic image - G06T 2207/30004: Biomedical image processing - G06T 2207/30204: Marker - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Calibration of parameters of an endoscopic or laparoscopic camera is conducted while the camera is in use capturing images of a surgical procedure at a surgical site. The surgical procedure is performed using a surgical instrument marked with a fiducial pattern. A processor receives image data from images captured by the camera and uses machine vision to detect points of the pattern, and carries out an optimization to determine the 3D pose of the surgical instrument relative to the camera and the camera parameters. ## Claims ### Claim 1 A surgical instrument having a distal portion positionable in a body cavity, the surgical instrument including an elongate shaft and a fiducial calibration pattern for use in estimating intrinsic camera parameters of a stereo endoscopic camera used at the surgical site, the fiducial calibration pattern positioned on the elongate shaft such that the fiducial calibration pattern is disposed within the body cavity and visible to a stereo endoscopic camera capturing images from within said body cavity during a surgical procedure. ### Claim 2 (depends on 1) The surgical instrument of claim 1, wherein the pattern comprises a plurality of polygons of known geometry. ### Claim 3 (depends on 2) The surgical instrument of claim 2, wherein the polygons are rectangles. ### Claim 4 (depends on 3) The surgical instrument of claim 3, wherein the pattern is a checkerboard pattern. ### Claim 5 (depends on 1) The surgical instrument of claim 1, wherein the surgical instrument includes a diagnostic or therapeutic element at a distal end of the shaft. ### Claim 6 (depends on 1) The surgical instrument of claim 1, wherein the fiducial calibration pattern extends 360 degrees around a circumference of the shaft. ### Claim 7 A method of calibrating a camera during surgery, comprising the steps of: positioning a camera with its distal end at a surgical treatment site within a body cavity; providing a surgical instrument having a fiducial calibration pattern thereon, and positioning the surgical instrument such that the fiducial calibration pattern is disposed within the body cavity; with the distal end of the camera within the body cavity, using the camera to capture a plurality of images of the surgical instrument within the body cavity; and based on locations of image fiducials in the plurality of images, and the known fiducials geometry, estimating the camera parameters. ### Claim 8 (depends on 7) The method of claim 7, wherein the fiducial calibration pattern is on the shaft of the instrument. ### Claim 9 (depends on 7) The method of claim 7, further comprising the step of using the surgical instrument to treat tissue at the treatment site while estimating the camera parameters. ### Claim 10 (depends on 7) The method of claim 7, further comprising the step of using the surgical instrument to diagnose a condition at the treatment site while estimating the camera parameters. ### Claim 11 (depends on 7) The method of claim 7, wherein estimating the camera parameters includes estimating the intrinsic camera parameters. ### Claim 12 (depends on 7) The method of claim 7, wherein estimating the camera parameters includes estimating the extrinsic camera parameters. ### Claim 13 (depends on 7) The method of claim 7, further including the step of using the camera to capture a plurality of surgical images of the surgical treatment site within the body cavity and displaying said plurality of surgical images in real time on a display. ### Claim 14 A surgical system comprising: a stereo endoscopic camera positionable in a body cavity; a surgical instrument positionable in the body cavity, the surgical instrument including an elongate shaft and a fiducial calibration pattern on the elongate shaft; at least one processor comprising at least one memory storing algorithms executable by the processor to: while the stereo endoscopic camera is positioned with its distal tip in the body cavity, receive images from the stereo endoscopic camera, the images comprising images captured from within the body cavity, wherein said images are of the fiducial calibration pattern within the body cavity, and using a location of scene points in the fiducial calibration pattern in a plurality of the images and a known geometry of the fiducial calibration pattern, estimating the parameters of the stereo endoscopic camera. ### Claim 15 (depends on 14) The system of claim 14, wherein the pattern comprises a plurality of polygons of known geometry. ### Claim 16 (depends on 15) The system claim 15, wherein the polygons are rectangles. ### Claim 17 (depends on 14) The system of claim 14, wherein the pattern is a checkerboard pattern. ### Claim 18 (depends on 14) The system of claim 14, wherein the surgical instrument includes a diagnostic or therapeutic element at a distal end of the shaft. ### Claim 19 (depends on 14) The system of claim 14, wherein the fiducial calibration pattern extends 360 degrees around a circumference of the shaft. ### Claim 20 (depends on 14) The system of claim 14, wherein the parameters include intrinsic parameters. ### Claim 21 (depends on 14) The system of claim 14, wherein the parameters include extrinsic parameters. ### Claim 22 (depends on 14) The method of claim 14, wherein the system further includes a display, wherein the camera is further for capturing a plurality of surgical images of a surgical treatment site within the body cavity for display in real time on the display.