# US 2025/0166798 A1 · Imaging system and methods displaying a fused multidimensional reconstructed image

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0166798 A1
- Document ID: US20250166798A1
- Status: Published application
- Kind: A1
- Application number: 19/028,390
- Filed: 2025-01-17
- Published: 2025-05-22
- Earliest claimed: 2012-08-30
- Assignees: Alcon Inc.
- Applicants: Alcon Inc.
- Inventors: Ashok Burton Tripathi; George Charles Polchin; Yen Ting Ng
- Web page: https://patentgrep.com/patent/US20250166798A1

## CPC

- G16H 30/40: for processing medical images, e.g. editing
- A61B 5/0077: {Devices for viewing the surface of the body, e.g. camera, magnifying lens}
- A61B 5/7425: {Displaying combinations of multiple images regardless of image source, e.g. displaying a reference anatomical image with a live image}
- A61B 5/743: {Displaying an image simultaneously with additional graphical information, e.g. symbols, charts, function plots}
- A61B 5/745: {using a holographic display}
- A61B 6/03: Computed tomography [CT]
- A61B 6/032: {Transmission computed tomography [CT]}
- A61B 6/037: {Emission tomography}
- A61B 90/20: Surgical microscopes characterised by non-optical aspects
- A61B 90/37: {Surgical systems with images on a monitor during operation}
- G06T 5/70: Denoising; Smoothing
- G06T 7/0014: {using an image reference approach}
- G06T 7/337: {involving reference images or patches}
- G06T 12/30: Image post-processing, e.g. metal artefact correction
- G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G16H 20/40: relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
- G16H 30/20: for handling medical images, e.g. DICOM, HL7 or PACS
- H04N 13/204: using stereoscopic image cameras (stereoscopic photography G03B35/00)
- H04N 13/30: Image reproducers (optical systems for producing stereoscopic or other three-dimensional [3D] effects G02B30/00)
- A61B 2090/365: {augmented reality, i.e. correlating a live optical image with another image}
- A61B 2090/3983: {Reference marker arrangements for use with image guided surgery}
- A61B 2576/00: Medical imaging apparatus involving image processing or analysis (A61B1/00009, A61B6/52 and A61B8/52 take precedence)
- A61B 2576/026: for the brain
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T 2207/10021: Stereoscopic video; Stereoscopic image sequence
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/10088: Magnetic resonance imaging [MRI]
- G06T 2207/20104: Interactive definition of region of interest [ROI]
- G06T 2207/20221: Image fusion; Image merging
- G06T 2207/30016: Brain
- G06T 2207/30196: Human being; Person
- G06T 2207/30204: Marker
- G06T 2210/41: Medical
- G06T 2210/62: Semi-transparency
- G06T 2219/028: Multiple view windows (top-side-front-sagittal-orthogonal)
- G06T 2219/2004: Aligning objects, relative positioning of parts
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A system, method, and apparatus for displaying a fused reconstructed image with a multidimensional image are disclosed. An example imaging system receives a selection corresponding to a portion of a displayed multidimensional visualization of a surgical site. At the selected portion of the multidimensional visualization, the imaging system displays a portion of a three-dimensional image which corresponds to the selected multidimensional visualization such that the displayed portion of the at least one of the three-dimensional image or model is fused with the displayed multidimensional visualization.

## Claims

### Claim 1

A method for visualizing a surgical site with an imaging system having an interface, a processor, a display device and a memory, the method comprising:
recording, through the use of at least one photosensor, a visualization of the surgical site;
creating a multidimensional visualization based on the recorded visualization of the surgical site, via a visualization generation system in the imaging system;
signaling respectively an increase and a decrease in a viewing depth of a currently displayed view of the multidimensional visualization in response to user input, via a foot-pedal device in the imaging system;
displaying the multidimensional visualization of the surgical site, via the display device; and
scrolling through different values of the viewing depth of the multidimensional visualization based on the user input to the foot-pedal device, via the processor.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
controlling a scale and/or transparency of the currently displayed view of the multidimensional reconstructed image, based on the user input to the foot-pedal device.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
controlling an orientation of the currently displayed view of the multidimensional reconstructed image, based on the user input to the foot-pedal device.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
positioning at least one of a template, image, and reference within the multidimensional reconstructed image, based on the user input to the foot-pedal device.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
indicating coordinates of the viewing depth of the multidimensional visualization through a depth meter in the imaging system.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
cycling through the different values of the viewing depth of the multidimensional visualization until a desired depth is displayed, via the user input to the foot-pedal device; and
updating the currently displayed view of the multidimensional reconstructed image to the desired depth.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
retrieving image data of the surgical site from at least one of a medical device and a server that stores the image data from a medical device, via the interface;
creating a three-dimensional image based on the image data, via the processor;
selecting a respective portion of the three-dimensional image that corresponds to the selected portion of the multidimensional visualization;
combining the respective portion of the three-dimensional image with the selected portion of the multidimensional visualization for a combined visualization; and
causing the display device to display the combined visualization.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
controlling a back-side depth of the multidimensional reconstructed image by moving forward a rear boundary of the multidimensional visualization until a desired depth is displayed, wherein the multidimensional visualization includes a view of a craniotomy.

### Claim 9 (depends on 1)

The method of claim 1, further comprising:
enabling a user to control alignment of the three-dimensional image with the multidimensional visualization of the surgical site through a voice activated control system in communication with the processor, wherein the image data includes at least one of pre-operative image data, perioperative image data, or intra-operative image data.

### Claim 10 (depends on 1)

The method of claim 1, further comprising:
combining the respective portion of the three-dimensional image with the selected portion of the multidimensional visualization by identifying first features of the multidimensional visualization, identifying second features of the three-dimensional image, and aligning the first features to correspond to the second features.

### Claim 11 (depends on 1)

The method of claim 1, further comprising:
combining the respective portion of the three-dimensional image with the selected portion of the multidimensional visualization by increasing a transparency of the selected portion of the multidimensional visualization and/or applying a smoothing function to a boundary between the respective portion of the three-dimensional image and the selected portion of the multidimensional visualization.

### Claim 12 (depends on 1)

The method of claim 1, further comprising:
transmitting an instruction to the display device specifying a location of the selected portion of the multidimensional visualization to which the respective portion of the reconstructed image is to be superimposed as a graphic.

### Claim 13 (depends on 1)

The method of claim 1, further comprising:
creating the three-dimensional image based on volumetric information of the image data, via the processor, the volumetric information including values defining an image at each point in a coordinate plane.

### Claim 14 (depends on 13)

The method of claim 13, further comprising:
creating the three-dimensional image by using at least one of density values and intensity values to define a tissue, via the processor, the volumetric information including the density values and the intensity values.

### Claim 15 (depends on 1)

The method of claim 1, further comprising combining the respective portion of the three-dimensional image with the selected portion of the multidimensional visualization by:
identifying a respective target structure within the multidimensional visualization;
identifying the respective target structure within the three-dimensional image;
aligning the three-dimensional image with the multidimensional visualization such that the respective target structure within the multidimensional visualization is aligned with the respective target structure within the three-dimensional image; and
identifying the respective portion of the three-dimensional image that is located at a same location as the selected portion of the multidimensional visualization.

### Claim 16 (depends on 18)

The method of claim 18, wherein the target structure includes a marked screw, further comprising:
registering the marked screw intra-operatively with at least one of an 0-arm imaging device and a C-arm imaging device.

### Claim 17

A non-transitory computer readable medium storing a set of computer instructions for visualizing a surgical site with an imaging system having an interface, a processor, a display device and a memory, the set of computer instructions being executable by a processor and comprising:
recording, through the use of at least one photosensor, a recorded visualization of the surgical site;
creating a multidimensional visualization based on the recorded visualization of the surgical site and the three-dimensional image, via a visualization generation system in the imaging system;
signaling respectively an increase and a decrease in a viewing depth of a currently displayed view of the multidimensional visualization in response to user input, via a foot-pedal device in the imaging system;
displaying the multidimensional visualization of the surgical site, via the display device; and
scrolling through different values of the viewing depth of the multidimensional visualization based on the user input to the foot-pedal device, via the processor.

### Claim 18 (depends on 17)

The non-transitory computer readable medium of claim 17, further comprising:
controlling a scale and/or transparency of the currently displayed view of the multidimensional reconstructed image, based on the user input to the foot-pedal device; and
controlling an orientation of the currently displayed view of the multidimensional reconstructed image, based on the user input to the foot-pedal device.

### Claim 19 (depends on 17)

The non-transitory computer readable medium of claim 17, further comprising:
cycling through the different values of the viewing depth of the multidimensional visualization until a desired depth is displayed, based on the user input to the foot-pedal device; and
updating the currently displayed view of the multidimensional reconstructed image to the desired depth.

### Claim 20 (depends on 17)

The non-transitory computer readable medium of claim 17, further comprising:
retrieving image data of the surgical site from at least one of a medical device and a server that stores the image data from a medical device, via the interface;
creating a three-dimensional image based on the image data, via the processor;
selecting a respective portion of the three-dimensional image that corresponds to a selected portion of the multidimensional visualization;
combining the respective portion of the three-dimensional image with the selected portion of the multidimensional visualization for a combined visualization; and
causing the display device to display the combined visualization.
