# US 2024/0005503 A1 · Method for processing medical images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0005503 A1
- Document ID: US20240005503A1
- Status: Published application
- Kind: A1
- Application number: 18/312,802
- Filed: 2023-05-05
- Published: 2024-01-04
- Earliest claimed: 2022-06-30
- Assignees: Siemens Healthcare GmbH
- Applicants: Siemens Healthcare GmbH
- Inventors: Yoshihisa Shinagawa; Halid Yerebakan; Gerardo Hermosillo Valadez; Mahesh Ranganath; Simon Allen-Raffl
- Web page: https://patentgrep.com/patent/US20240005503A1

## CPC

- G06T 7/0014: {using an image reference approach}
- G06T 7/0016: {involving temporal comparison}
- G06T 7/13: Edge detection
- G06T 7/33: using feature-based methods
- G06T 7/337: {involving reference images or patches}
- G06T 7/73: using feature-based methods
- G06T 11/00: Two-dimensional [2D] image generation
- A61B 5/7425: {Displaying combinations of multiple images regardless of image source, e.g. displaying a reference anatomical image with a live image}
- G06F 16/35: Clustering; Classification
- G06T 2207/10072: Tomographic images
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/20101: Interactive definition of point of interest, landmark or seed
- G06T 2207/30096: Tumor; Lesion
- G06T 2207/30204: Marker
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A framework for processing medical images. The framework may include receiving a target medical image, a reference medical image and at least one marker associated with a location in the reference medical image. A corresponding location of the at least one marker is determined in the target medical image. The target medical image is overlaid with the at least one marker at the determined corresponding location to provide an overlaid image. Display data is generated to cause a display device to display the overlaid image.

## Claims

### Claim 1

A computer-implemented method for processing medical images,
the method comprising:
a) receiving a target medical image;
b) receiving a reference medical image;
c) receiving at least one marker associated with a location (x, y, z) in the reference medical image;
d) determining a corresponding location (x′, y′, z′) of the at least one marker in the target medical image;
e) overlaying the target medical image with the at least one marker at the corresponding location (x′, y′, z′) to provide an overlaid image; and
f) generating display data to cause a display device to display the overlaid image.

### Claim 2 (depends on 1)

The method of claim 1 wherein step d) comprises performing a registration between the target medical image and reference medical image.

### Claim 3 (depends on 2)

The method of claim 2, wherein the registration comprises:
generating a first descriptor by sampling image data in the reference medical image encompassing the location (x, y, z) of the at least one marker;
generating second descriptors by sampling image data for each of a plurality of candidate locations in the target medical image;
calculating, for each of the plurality of candidate locations, an image data similarity metric indicating a degree of similarity between the first descriptor and the second descriptors;
selecting a candidate location from among the plurality of candidate locations based on the image data similarity metric; and
determining the corresponding location (x′, y′, z′) in the target medical image based on the selected candidate location.

### Claim 4 (depends on 2)

The method of claim 2, wherein step d) comprises:
determining, based on the registration, an initial corresponding location (x′, y′, z′); and
determining a refined corresponding location (x″, y″, z″).

### Claim 5 (depends on 4)

The method of claim 4 wherein determining the refined corresponding location (x″, y″, z″) comprises:
selecting, in the target medical image, a neighborhood region encompassing the initial corresponding location (x′, y′, z′);
generating a descriptor by sampling image data encompassing the initial corresponding location (x′, y′, z′); and
applying a trained neural network to the neighborhood region or to the descriptor to output the refined corresponding location (x″, y″, z″).

### Claim 6 (depends on 5)

The method of claim 5 further comprising:
applying an edge detection algorithm to the neighborhood region or to the descriptor to output the refined corresponding location (x″, y″, z″).

### Claim 7 (depends on 5)

The method of claim 5 wherein the trained neural network is a regressor.

### Claim 8 (depends on 7)

The method of claim 7 wherein the regressor is trained prior to step d) using a self-supervised training method.

### Claim 9 (depends on 7)

The method of claim 7 wherein the regressor is based on deep learning.

### Claim 10 (depends on 7)

The method of claim 7 wherein the regressor is trained using lesion marker data sets.

### Claim 11 (depends on 1)

The method of claim 1 wherein the at least one marker comprises two end points, wherein the target medical image and reference medical image are slices and the two end points are located within a respective one of the slices.

### Claim 12 (depends on 1)

The method of claim 1 wherein the at least one marker is associated with a region in the target medical image or reference medical image, wherein all pixels in said region having a value above, below or between a value defined prior to step d).

### Claim 13 (depends on 1)

The method of claim 1, wherein the at least one marker may comprises:
two end points and a distance line connecting the two end points;
two end points and a circle, a radius of which is derived from the two end points, wherein the circle is centered between the two end points; and
a crosshair.

### Claim 14 (depends on 1)

The method of claim 1 wherein at least one property of the at least one marker overlaid in step e) is automatically adjusted compared to at least one property of the at least on marker received in step c) depending on a change in at least one property of the target medical image compared to the reference medical image.

### Claim 15 (depends on 14)

The method of claim 14 wherein the at least one property of the at least one marker is a size of the at least one marker when output on the display device.

### Claim 16 (depends on 14)

The method of claim 14 wherein the at least one property of the target medical image compared to the reference medical image is a size of an anatomical object within the target medical image and reference medical image to which the at least one marker refers.

### Claim 17

A device for processing medical images, comprising:
one or more non-transitory computer-readable media for storing a module;
a first receiving unit which is configured to receive at least one target medical image and at least one reference medical image captured by a medical imaging unit;
a second receiving unit which is configured to at least one marker associated with a location (x, y, z) in the reference medical image; and
one or more processing units in communication with the one or more non-transitory computer-readable media, the one or more processing units being operative with the module to perform steps including
a) determining a corresponding location (x′, y′, z′) of the at least one marker in the target medical image,
b) overlaying the target medical image with the at least one marker at the corresponding location (x′, y′, z′) to provide an overlaid image, and
c) generating display data to cause a display device to display the overlaid image.

### Claim 18 (depends on 17)

The device of claim 17 wherein the at least one marker is associated with a region in the target medical image or reference medical image, wherein all pixels in said region having a value above, below or between a value defined prior to step a).

### Claim 19 (depends on 17)

The device of claim 17 wherein at least one property of the at least one marker overlaid in step b) is automatically adjusted compared to at least one property of the at least on marker received in step a) depending on a change in at least one property of the target medical image compared to the reference medical image.

### Claim 20

One or more non-transitory computer-readable media embodying instructions executable by machine to perform steps comprising:
a) receiving a target medical image;
b) receiving a reference medical image;
c) receiving at least one marker associated with a location (x, y, z) in the reference medical image;
d) determining a corresponding location (x′, y′, z′) of the at least one marker in the target medical image;
e) overlaying the target medical image with the at least one marker at the corresponding location (x′, y′, z′) to provide an overlaid image; and
f) generating display data to cause a display device to display the overlaid image.
