# US 2024/0389837 A1 · Method of Optical Image Stabilization and Active Alignment of Sensors in Endoscopic Camera Systems

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0389837 A1
- Document ID: US20240389837A1
- Status: Published application
- Kind: A1
- Application number: 18/792,980
- Filed: 2024-08-02
- Published: 2024-11-28
- Earliest claimed: 2022-09-30
- Assignees: KARL STORZ Imaging, Inc.
- Applicants: KARL STORZ Imaging, Inc.
- Inventors: Christopher Zimmer
- Web page: https://patentgrep.com/patent/US20240389837A1

## CPC

- A61B 1/00163: {Optical arrangements (A61B1/002, A61B1/06 take precedence)}
- A61B 1/00009: {of image signals during a use of endoscope}
- A61B 1/0005: {combining images e.g. side-by-side, superimposed or tiled}
- A61B 1/00057: {provided with means for testing or calibration}
- A61B 1/00066: {Proximal part of endoscope body, e.g. handles (A61B1/0052 takes precedence)}
- A61B 1/00096: {Optical elements}
- G02B 23/2484: {Arrangements in relation to a camera or imaging device (processing or control of video signals generated by an endoscope H04N23/60, H04N23/70)}
- H04N 23/555: {for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes}
- A61B 1/00186: {with imaging filters}
- A61B 1/042: {characterised by a proximal camera, e.g. a CCD camera}
- A61B 1/043: {for fluorescence imaging}
- A61B 1/051: {Details of CCD assembly}
- H04N 23/45: for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Systems and methods are provided that enable optical image stabilization and image sensor alignments. By enabling image sensors to be adjusted based on identified misalignments, users of the systems and methods disclosed herein can adjust imaging during the course of a surgery or surgical procedure, without needing to stop the surgery or surgical procedure to manually realign optical components. The optical image stabilization allows for adjustment of sensor components to compensate for shifts or other movement of imaging devices and to maintain focus on a target object. The optical image stabilization also includes digital masks that can be overlaid on an image to block distracting mask shifting.

## Claims

### Claim 1

A method, comprising:
receiving light at an endoscope;
splitting the light into a first light path and a second light path with an optical component;
determining a misalignment between a first sensor and a second sensor, the first sensor disposed on a chassis in the endoscope and aligned with the first light path, and the second sensor disposed proximate the optical component in the endoscope and aligned with the second light path;
determining a first change in pose of the first sensor to compensate for the misalignment, the determination made based at least partially on a registration between the first sensor and the second sensor; and
causing the first change in pose of the first sensor.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
receiving a first video stream from the first sensor and a second video stream from the second sensor.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
performing feature detection on the first video stream and the second video stream; and
determining, based on the feature detection, an amount of offset between the first video stream and the second video stream.

### Claim 4 (depends on 1)

The method of claim 1, wherein the endoscope comprises an optical assembly that channels the light into the optical component, and wherein the optical assembly comprises at least one of one or more mirrors, one or more windows, one or more polarizers, one or more lenses, and one or more filters.

### Claim 5 (depends on 1)

The method of claim 1, wherein the first sensor is disposed in a sensor module, the sensor module comprising:
a plurality of magnets disposed proximate the first sensor;
a plurality of electromagnets disposed within an interior of the sensor module; and
a position sensor disposed within the interior of the sensor module.

### Claim 6 (depends on 5)

The method of claim 5, wherein causing the first change in pose of the first sensor includes:
passing current through the plurality of electromagnets such that an electromagnetic interaction between the plurality of electromagnets and the plurality of magnets causes the first sensor to move.

### Claim 7 (depends on 1)

The method of claim 1, comprising the further step of passing the light collected by the endoscope to a connected camera head, the camera head containing the first sensor and the second sensor.

### Claim 8 (depends on 7)

The method of claim 7, wherein the step of splitting the light into a first light path and a second light path with an optical component is performed within the camera head.

### Claim 9 (depends on 2)

The method of claim 2, comprising the further steps of receiving information associated with a movement of the endoscope;
determining, based on the information, a movement of the first sensor to compensate for the movement of the endoscope;
determining, based on the movement of the endoscope, a change in position of an object depicted in the video stream;
shifting, based on the movement of the endoscope, a digital mask; and
overlaying, on an exported video stream, the digital mask.

### Claim 10 (depends on 6)

The method of claim 6, wherein the change in pose of the first sensor is performed iteratively, and wherein the current passed through the plurality of electromagnets is adjusted based on a reading from the position sensor.

### Claim 11 (depends on 9)

The method of claim 9, wherein the digital mask is smaller than an endoscopic mask, the endoscopic mask being an artifact of one or more optical elements of the endoscope an apparatus connected thereto, and rendering the exported video stream on a display such that the video stream does not change when the endoscope moves and the digital mask is overlaid on the video stream.

### Claim 12

An apparatus, comprising:
a camera head configured to receive light from an endoscope attached thereto, the camera head comprising:
an optical component configured to split light into two or more light paths;
a first sensor disposed on a chassis and aligned with a first light path of the two or more light paths; and
a second sensor disposed proximate the optical component and aligned with a second light path of the two or more light paths, wherein the first sensor is configured to move relative to at least one of the optical component and the second sensor, and wherein the movement is orthogonal to an optical axis of the optical component.

### Claim 13 (depends on 12)

The apparatus of claim 12, wherein the optical component is a beam splitter.

### Claim 14 (depends on 13)

The apparatus of claim 13, wherein the beam splitter comprises a prism.

### Claim 15 (depends on 12)

The apparatus of claim 12, further comprising:
a processor; and
a memory storing data thereon that, when processed by the processor, cause the processor to:
receive a first video stream from the first sensor;
receive information about a first movement of the endoscope;
determine, based on the first movement, a change in pose of the first sensor to compensate for the first movement of the endoscope; and
cause the change in pose of the first sensor.

### Claim 16 (depends on 12)

The apparatus of claim 12, wherein the first sensor is disposed in a sensor module, the sensor module comprising:
a plurality of magnets disposed proximate the first sensor;
a plurality of electromagnets disposed within an interior of the sensor module; and
a position sensor disposed within the interior of the sensor module.

### Claim 17 (depends on 16)

The apparatus of claim 16, wherein a processor causes a change in pose of the first sensor by passing current through the plurality of electromagnets such that an electromagnetic interaction between the plurality of electromagnets and the plurality of magnets causes the first sensor to move.

### Claim 18 (depends on 17)

The apparatus of claim 17, wherein the processor causes the change in pose of the first sensor iteratively, and wherein the current passed through the plurality of electromagnets is adjusted based on a reading from the position sensor.
