# US 2023/0404513 A1 · Multi-modal imaging alignment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0404513 A1
- Document ID: US20230404513A1
- Status: Published application
- Kind: A1
- Application number: 18/241,428
- Filed: 2023-09-01
- Published: 2023-12-21
- Earliest claimed: 2018-03-19
- Assignees: Koninklijke Philips N.V.
- Applicants: Koninklijke Philips N.V.
- Inventors: FRANCOIS GUY GERARD MARIE VIGNON; MOLLY LARA FLEXMAN; AMEET KUMAR JAIN; NIELS NIJHOF
- Web page: https://patentgrep.com/patent/US20230404513A1

## CPC

- A61B 6/5247: {combining images from an ionising-radiation diagnostic technique and a non-ionising radiation diagnostic technique, e.g. X-ray and ultrasound}
- A61B 6/12: Arrangements for detecting or locating foreign bodies
- A61B 6/4417: {related to combined acquisition of different diagnostic modalities}
- A61B 8/0841: {for locating instruments}
- A61B 8/0883: {for diagnosis of the heart}
- A61B 8/12: in body cavities or body tracts, e.g. by using catheters
- A61B 8/4245: {involving determining the position of the probe, e.g. with respect to an external reference frame or to the patient}
- A61B 8/4416: {related to combined acquisition of different diagnostic modalities, e.g. combination of ultrasound and X-ray acquisitions}
- A61B 8/5261: {combining images from different diagnostic modalities, e.g. ultrasound and X-ray}
- A61B 8/5276: {due to motion}
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)

## Abstract

A controller for maintaining alignment of X-Ray imagery and ultrasound imagery includes a memory that stores instructions, and a processor that executes the instructions. When executed by the processor, the instructions cause the controller to execute a process that includes receiving data from an X-Ray system used to perform X-Ray imaging, and receiving data from an ultrasound imaging probe used to perform ultrasound imaging. The process executed by the controller also includes registering imagery based on X-Rays to imagery from the ultrasound imaging probe based on an X-Ray image of the ultrasound imaging probe among the imagery based on X-Rays, and detecting, from the data from the ultrasound imaging probe, movement of the ultrasound imaging probe.

## Claims

### Claim 1

A controller for maintaining alignment of X-Ray imagery and ultrasound imagery, the controller comprising:
a processor and memory, the processor configured to:
receive X-Ray imaging data from an X-Ray system, wherein the X-Ray imaging data comprises X-Ray imagery of an anatomical region,
receive ultrasound imaging data from an ultrasound imaging probe, wherein the ultrasound imaging data comprises ultrasound imagery of the anatomical region,
register, in a common coordinate system, the X-Ray imagery to the ultrasound imagery,
detect movement of the ultrasound imaging probe based on a change in position of the ultrasound imaging probe between a new ultrasound data set and a previous ultrasound data set received from the ultrasound imaging probe, and
re-align the ultrasound imagery and the X-Ray imagery by adjusting the ultrasound imagery or the X-Ray imagery relative to the common coordinate system to compensate for the detected movement of the ultrasound imaging probe.

### Claim 2 (depends on 1)

The controller of claim 1, wherein the processor is further configured to:
determine that the movement of the ultrasound imaging probe exceeds a predetermined threshold, and
generate a notification based on detecting the movement of the ultrasound imaging probe.

### Claim 3 (depends on 2)

The controller of claim 2, wherein:
the predetermined threshold varies among a plurality of predetermined thresholds, and
the notification varies based on an amount of the movement of the ultrasound imaging probe.

### Claim 4 (depends on 1)

The controller of claim 1, wherein the processor is further configured to:
adjust the ultrasound imagery or the X-Ray imagery relative to the common coordinate system includes at least one of shifting image information one or more axis or rotating image information about one or more axis.

### Claim 5 (depends on 1)

The controller of claim 1, wherein the processor is further configured to:
measure the amount of the movement of the ultrasound imaging probe, and
wherein the adjusting of the ultrasound imagery or the X-Ray imagery is to correct a registration mismatch between the X-Ray imagery and the ultrasound imagery based on the measured amount of movement.

### Claim 6 (depends on 1)

The controller of claim 1, wherein the processor is further configured to:
detect the movement of the ultrasound imaging probe based on analysis of a cineloop from the ultrasound imaging probe.

### Claim 7 (depends on 1)

The controller of claim 1, wherein the processor is further configured to:
identify a region in which an interventional medical device is moved in the cineloop from the ultrasound imaging probe, and
exclude movement within the region from detection of the movement of the ultrasound imaging probe.

### Claim 8 (depends on 1)

The controller of claim 1, wherein the processor is further configured to:
identify a three-dimensional cardiac cycle over time that affects the ultrasound imaging probe as a dataset of a four-dimensional cardiac cycle, and
detect movement of the ultrasound imaging probe between cycles of the four-dimensional cardiac cycle using the dataset.

### Claim 9 (depends on 1)

The controller of claim 1, wherein the processor is further configured to:
identify a respiratory cycle that affects the ultrasound imaging probe, and
identify and remove from the detected movement periodicity due to respiration in the respiratory cycle.

### Claim 10 (depends on 9)

The controller of claim 9, wherein the processor is further configured to:
compensate for respiratory motion based on input received from an external respiration device.

### Claim 11 (depends on 1)

The controller of claim 1, wherein the processor is further configured to:
detect the movement of the ultrasound imaging probe based on measurements of an external sensor.

### Claim 12

A method for maintaining alignment of X-Ray imagery and ultrasound imagery, the method comprising:
receiving X-Ray imaging data from an X-Ray system, wherein the X-Ray imaging data comprises X-Ray imagery of an anatomical region;
receiving ultrasound imaging data from an ultrasound imaging probe, wherein the ultrasound imaging data comprises ultrasound imagery of the anatomical region;
registering, in a common coordinate system, the X-Ray imagery to the ultrasound imagery;
detecting movement of the ultrasound imaging probe based on a change in position of the ultrasound imaging probe between a new ultrasound data set and a previous ultrasound data set received from the ultrasound imaging probe; and
re-aligning the ultrasound imagery and the X-Ray imagery by adjusting the ultrasound imagery or the X-Ray imagery relative to the common coordinate system to compensate for the detected movement of the ultrasound imaging probe.

### Claim 13 (depends on 12)

The method of claim 12, wherein adjusting the ultrasound imagery or the X-Ray imagery relative to the common coordinate system includes at least one of shifting image information one or more axis or rotating image information about one or more axis.

### Claim 14 (depends on 12)

The method of claim 12, further comprising:
determining that the movement of the ultrasound imaging probe exceeds a predetermined threshold; and
generating a notification based on detecting the movement of the ultrasound imaging probe.

### Claim 15 (depends on 14)

The method of claim 14, further comprising:
identifying a respiratory cycle that affects the ultrasound imaging probe; and
identifying and remove from the detected movement periodicity due to respiration in the respiratory cycle.

### Claim 16 (depends on 12)

The method of claim 12, further comprising:
identifying a region in which an interventional medical device is moved in a cineloop from the ultrasound imaging probe; and
exclude movement within the region from the detected movement of the ultrasound imaging probe.

### Claim 17

A non-transitory computer-readable storage medium having stored a computer program comprising instructions, which, when executed by a processor, cause the processor to:
receive X-Ray imaging data from an X-Ray system, wherein the X-Ray imaging data comprises X-Ray imagery of an anatomical region;
receive ultrasound imaging data from an ultrasound imaging probe, wherein the ultrasound imaging data comprises ultrasound imagery of the anatomical region;
register, in a common coordinate system, the X-Ray imagery to the ultrasound imagery;
detect movement of the ultrasound imaging probe based on a change in position of the ultrasound imaging probe between a new ultrasound data set and a previous ultrasound data set received from the ultrasound imaging probe; and
re-align the ultrasound imagery and the X-Ray imagery by adjusting the ultrasound imagery or the X-Ray imagery relative to the common coordinate system to compensate for the detected movement of the ultrasound imaging probe.

### Claim 18 (depends on 17)

The non-transitory computer-readable storage medium of claim 17, wherein adjusting the ultrasound imagery or the X-Ray imagery relative to the common coordinate system includes at least one of shifting image information one or more axis or rotating image information about one or more axis.

### Claim 19 (depends on 17)

The non-transitory computer-readable storage medium of claim 17, wherein the instructions, when executed by the processor, further cause the processor to:
measure the amount of the movement of the ultrasound imaging probe, and
wherein the adjusting of the ultrasound imagery or the X-Ray imagery is to correct a registration mismatch between the X-Ray imagery and the ultrasound imagery based on the measured amount of movement.

### Claim 20 (depends on 17)

The non-transitory computer-readable storage medium of claim 17, wherein the instructions, when executed by the processor, further cause the processor to:
identify a region in which an interventional medical device is moved in the cineloop from the ultrasound imaging probe; and
exclude movement within the region from the detected movement of the ultrasound imaging probe.
