# US 12,178,401 B2 · Systems and methods for characterization of an endoscope and automatic calibration of an endoscopic camera system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,178,401 B2
- Document ID: US12178401B2
- Status: Granted
- Kind: B2
- Application number: 17/762,179
- Filed: 2020-10-07
- Published: 2022-12-08
- Granted: 2024-12-31
- Earliest claimed: 2019-10-07
- Assignees: S&N Orion Prime, S.A.
- Applicants: S&N Orion Prime, S.A.
- Inventors: João Pedro De Almeida Barreto; Carolina Dos Santos Raposo; Michel Goncalves Almeida Antunes; Rui Jorge Melo Teixeira
- Web page: https://patentgrep.com/patent/US12178401B2

## CPC

- A61B 1/00179: {for off-axis viewing}
- A61B 1/00009: {of image signals during a use of endoscope}
- A61B 1/000094: {extracting biological structures}
- A61B 1/000096: {using artificial intelligence}
- A61B 1/0005: {combining images e.g. side-by-side, superimposed or tiled}
- A61B 1/00183: {for variable viewing angles}
- A61B 1/00188: {with focusing or zooming features}
- G02B 23/2484: {Arrangements in relation to a camera or imaging device (processing or control of video signals generated by an endoscope H04N23/60, H04N23/70)}
- G06T 3/18: Image warping, e.g. rearranging pixels individually
- G06T 7/13: Edge detection
- G06T 7/33: using feature-based methods
- G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T 15/205: {Image-based rendering}
- G06T 2207/10: Image acquisition modality
- G06T 2207/10068: Endoscopic image
- G06T 2210/41: Medical
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Systems and methods for determining the calibration of an endoscopic camera consisting in a camera-head equipped with exchangeable, rotatable optics (the rigid endoscope), which is accomplished with no user intervention in the Operating Room by first characterizing the rigid endoscope through a set of parameters Φ and then using this lens descriptor Φ as input in a real-time software that processes the images acquired by an arbitrary camera-head equipped with the rigid endoscope, to provide the calibration of the complete endoscopic camera arrangement at every frame time instant, irrespective of the relative rotation of the lens scope with respect to camera-head or zoom settings. Also disclosed is an image software method that detects if the lens descriptor is not compatible with the rigid endoscope in use to prevent errors and warn the user about faulty situations.

## Claims

### Claim 1

A method for obtaining a lens descriptor for a rigid endoscope with a Field Stop Mask (FSM), the method comprising:
with the rigid endoscope installed in a characterization camera including the rigid endoscope and a camera head:
acquiring, using the characterization camera, K calibration images at N angular positions between the rigid endoscope and the camera head;
estimating first calibration parameters of the characterization camera at a reference angular position; and
determining the lens descriptor for the rigid endoscope based on the first calibration parameters estimated for the characterization camera at the reference angular position, wherein the lens descriptor includes second calibration parameters different from the first calibration parameters; and
with the rigid endoscope installed in an endoscopic camera different from the characterization camera, calibrating the endoscopic camera using the second calibration parameters contained in the lens descriptor for the rigid endoscope.

### Claim 2 (depends on 1)

The method of claim 1, further comprising computing a normalized focal length {dot over (f)} from {dot over (f)}=f/r, with r being a distance between center C0=[Cx, Cy, 1]T and notch P0, and obtaining a normalized principal point {dot over (O)} and a normalized rotation center {dot over (Q)} by computing {dot over (O)}=AO and {dot over (Q)}=AQ, respectively, with A=
A
=
[
cos
⁢
β
-
sin
⁢
β
-
C
x
⁢
cos
⁢
β
+
C
y
⁢
sin
⁢
β
sin
⁢
β
cos
⁢
β
-
C
x
⁢
sin
⁢
β
+
C
y
⁢
cos
⁢
β
0
0
r
]
and β being an angle between line CP and a down direction.

### Claim 3 (depends on 1)

The method of claim 1, further comprising calibrating the endoscopic camera by:
acquiring an image frame i with the endoscopic camera;
detecting a boundary center Ci and a notch Pi in the image frame i; and
estimating a focal length f, a rotation center Q, and a principal point O according to the boundary center Ci, the notch Pi, and the second calibration parameters contained in the lens descriptor for the rigid endoscope, wherein the second calibration parameters include a normalized focal length {dot over (f)}, a normalized principal point {dot over (O)}, and a normalized rotation center {dot over (Q)}.

### Claim 4 (depends on 3)

The method of claim 3, further comprising determining a radius r of the image frame.

### Claim 5 (depends on 4)

The method of claim 4, wherein the focal length f, the principal point O and the rotation center Q are computed by f=r{dot over (f)}, O=B{dot over (O)} and Q=B{dot over (Q)}, respectively, with B=
[
r
⁢
cos
⁢
α
r
⁢
sin
⁢
α
C
x
-
r
⁢
sin
⁢
α
r
⁢
cos
⁢
α
C
y
0
0
1
]
and α being the angle between line ∥CiPi∥ and the down direction.

### Claim 6 (depends on 3)

The method of claim 3, wherein the lens descriptor is obtained by one or more of:
loading information into a camera control unit (CCU) from a database;
reading a QR code;
obtaining information from a USB flash drive;
receiving manual input of information from a user;
reading engravings in the FSM;
obtaining information from an RFID tag; or
obtaining information over an internet connection.

### Claim 7 (depends on 3)

The method of claim 3, wherein frame i comprises two or more frames, wherein the rotation center Q is determined using one or more of:
three or more boundary centers Ci;
three or more notches Pi; or
at least one boundary center Ci and at least one notch Pi.

### Claim 8 (depends on 1)

The method of claim 1, wherein estimating the first calibration parameters includes estimating the first calibration parameters based on rotation of a principal point O around a rotation center Q in the K calibration images.

### Claim 9 (depends on 8)

The method of claim 8, wherein the first calibration parameters include a focal length f, a distortion ξ, a principal point O0, and a rotation center Q of the characterization camera at the reference angular position.

### Claim 10 (depends on 9)

The method of claim 9, wherein the second calibration parameters contained in the lens descriptor include a normalized focal length {dot over (f)}, a distortion ξ, a normalized principal point {dot over (O)}, and a normalized rotation center {dot over (Q)}.

### Claim 11 (depends on 10)

The method of claim 10, further comprising detecting a boundary with a center C0 and a notch P0 at the reference angular position, wherein determining the lens descriptor includes determining the normalized focal length {dot over (f)}, the distortion ξ, the normalized principal point {dot over (O)}, and the normalized rotation center {dot over (Q)} using the center C0, the notch P0, the focal length f, the principal point O0, and the rotation center Q.

### Claim 12 (depends on 1)

The method of claim 1, wherein calibrating the endoscopic camera includes calibrating the endoscopic camera at least one of (i) at the reference angular position and (ii) in accordance with rotation of the rigid endoscope.