# US 2025/0095152 A1 · Spatial binding of imaging data from multiple modalities

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0095152 A1
- Document ID: US20250095152A1
- Status: Published application
- Kind: A1
- Application number: 18/828,271
- Filed: 2024-09-09
- Published: 2025-03-20
- Earliest claimed: 2023-09-15
- Assignees: Alcon Inc.
- Applicants: Alcon, Inc.
- Inventors: Vignesh Suresh; Lu Yin; Ramesh Sarangapani
- Web page: https://patentgrep.com/patent/US20250095152A1

## CPC

- G06T 7/0014: {using an image reference approach}
- G06F 18/256: {of results relating to different input data, e.g. multimodal recognition}
- G06T 7/337: {involving reference images or patches}
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- G06V 10/751: {Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching}
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/806: {of extracted features}
- G06V 10/82: using neural networks
- G06V 20/70: Labelling scene content, e.g. deriving syntactic or semantic representations
- G06V 40/18: Eye characteristics, e.g. of the iris
- G16H 30/40: for processing medical images, e.g. editing
- G06T 2207/10101: Optical tomography; Optical coherence tomography [OCT]
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/20212: Image combination
- G06T 2207/30041: Eye; Retina; Ophthalmic
- G06T 2207/30096: Tumor; Lesion
- G06V 2201/03: Recognition of patterns in medical or anatomical images
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

A system for spatially binding images from multiple modalities includes a controller having at least one processor and at least one non-transitory, tangible memory. The controller is adapted to receive first and second imaging datasets of a target site from a first and a second modality. The controller is adapted to extract a first feature set in the first imaging dataset, via a first neural network. A second feature set is extracted from the second imaging dataset, via a second neural network. Feature pairs are generated by matching respective datapoints in the first feature set and the second feature set. The controller is adapted to determine a coordinate transformation between the feature pairs and generate at least one spatially bound image of the target site based in part on the coordinate transformation.

## Claims

### Claim 1

A system comprising:
a controller having at least one processor and at least one non-transitory, tangible memory on which instructions are recorded for a method of spatially binding imaging data, execution of the instructions by the processor causing the controller to:
receive a first imaging dataset of a target site from a first modality and a second imaging dataset of the target site from a second modality;
extract a first feature set from the first imaging dataset, via a first neural network;
extract a second feature set from the second imaging dataset, via a second neural network;
generate feature pairs by matching a respective datapoint in the first feature set with the respective datapoint in the second feature set;
determine a coordinate transformation between the feature pairs; and
generate at least one spatially bound image of the target site based in part on the first imaging dataset, the second imaging dataset and the coordinate transformation.

### Claim 2 (depends on 1)

The system of claim 1, wherein the target site is an eye.

### Claim 3 (depends on 1)

The system of claim 1, wherein:
the first feature set is a limited set that is not representative of information captured by the first modality; and
the second feature is a representation of an ocular region captured by the second modality.

### Claim 4 (depends on 1)

The system of claim 1, wherein the controller is configured to select at least one region of interest in the first imaging dataset, the first feature set being extracted from the at least one region of interest.

### Claim 5 (depends on 1)

The system of claim 1, wherein the first imaging dataset includes data obtained by scanning a plurality of source wavelengths.

### Claim 6 (depends on 1)

The system of claim 1, wherein the first modality is multispectral imaging, and the second modality is optical coherence tomography (“OCT”).

### Claim 7 (depends on 6)

The system of claim 6, wherein:
the first imaging dataset is captured by an OCT device, the target site being an eye; and
the at least one spatially bound image extends a peripheral portion of the first imaging dataset, enabling visualization of one or more structures posterior to an iris.

### Claim 8 (depends on 6)

The system of claim 6, wherein the first neural network is a multilayer perceptron.

### Claim 9 (depends on 8)

The system of claim 8, wherein the second neural network is a convolutional neural network.

### Claim 10 (depends on 1)

The system of claim 1, wherein the first modality is fluorescent angiography, and the second modality is optical coherence tomography.

### Claim 11 (depends on 1)

The system of claim 1, wherein the first feature set includes the respective feature points indicative of degenerative disease.

### Claim 12 (depends on 1)

The system of claim 1, wherein:
the first imaging dataset includes a plurality of scans;
the controller is configured to identify and isolate a pathological region as being within one of the plurality of scans or in-between two of the plurality of scans; and
the controller is configured to add at least one annotation over the at least one spatially bound image, the at least one annotation indicating the pathological region.

### Claim 13 (depends on 1)

The system of claim 1, wherein the controller is adapted to selectively transfer respective annotations made on the first imaging dataset to the second imaging dataset, and from the second imaging dataset to the first imaging dataset.

### Claim 14

A system comprising:
a controller having at least one processor and at least one non-transitory, tangible memory on which instructions are recorded for a method of spatially binding imaging data, execution of the instructions by the processor causing the controller to:
receive a first imaging dataset of an eye from a first modality and a second imaging dataset of the eye from a second modality, the first modality being multispectral imaging, and the second modality being optical coherence tomography;
select at least one region of interest in the first imaging dataset;
extract a first feature set from the at least one region of interest, via a first neural network;
extract a second feature set from the second imaging dataset, via a second neural network;
generate feature pairs by matching a respective datapoint in the first feature set with the respective datapoint in the second feature set; and
determine a coordinate transformation between the feature pairs and generate at least one spatially bound image of the eye based in part on the coordinate transformation.

### Claim 15 (depends on 14)

The system of claim 14, wherein:
the first feature set is a limited set that is not representative of information captured by the first modality; and
the second feature is a representation of an ocular region captured by the second modality.

### Claim 16 (depends on 14)

The system of claim 14, wherein the first neural network is a multilayer perceptron, and the second neural network is a convolutional neural network.

### Claim 17 (depends on 14)

The system of claim 14, wherein:
the first imaging dataset includes a plurality of scans;
the controller is configured to identify and isolate a pathological region as being within one of the plurality of scans or in-between two of the plurality of scans; and
the controller is configured to add at least one annotation over the at least one spatially bound image, the at least one annotation indicating the pathological region.

### Claim 18 (depends on 14)

The system of claim 14, wherein the controller is adapted to selectively transfer respective annotations made on the first imaging dataset to the second imaging dataset, and from the second imaging dataset to the first imaging dataset.