# US 10,945,708 B2 · Method and apparatus for registration of medical images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,945,708 B2
- Document ID: US10945708B2
- Status: Granted
- Kind: B2
- Application number: 14/597,290
- Filed: 2015-01-15
- Published: 2015-10-15
- Granted: 2021-03-16
- Earliest claimed: 2014-04-14
- Assignees: Samsung Electronics Co., Ltd.; Korea Advanced Institute of Science and Technology
- Applicants: Samsung Electronics Co., Ltd.; Korea Advanced Institute of Science and Technology
- Inventors: Youngkyoo Hwang; Jungbae Kim; Youngtaek Oh; Wonchul Bang; Jong Beom Ra; Woo Hyun Nam; Chijun Weon
- Web page: https://patentgrep.com/patent/US10945708B2

## CPC

- A61B 8/5261: {combining images from different diagnostic modalities, e.g. ultrasound and X-ray}
- A61B 5/1128: {using image analysis (A61B5/1127 takes precedence)}
- A61B 5/113: occurring during breathing
- A61B 5/7289: {Retrospective gating, i.e. associating measured signals or images with a physiological event after the actual measurement or image acquisition, e.g. by simultaneously recording an additional physiological signal during the measurement or image acquisition}
- A61B 6/032: {Transmission computed tomography [CT]}
- A61B 6/4417: {related to combined acquisition of different diagnostic modalities}
- A61B 6/486: {involving generating temporal series of image data}
- A61B 6/5247: {combining images from an ionising-radiation diagnostic technique and a non-ionising radiation diagnostic technique, e.g. X-ray and ultrasound}
- A61B 6/5288: {involving retrospective matching to a physiological signal}
- A61B 8/463: {characterised by displaying multiple images or images and diagnostic data on one display}
- G06T 7/33: using feature-based methods
- A61B 6/037: {Emission tomography}
- A61B 2505/05: Surgical care
- A61B 2576/02: specially adapted for a particular organ or body part
- G01R 33/5608: {Data processing and visualization specially adapted for MR, e.g. for feature analysis and pattern recognition on the basis of measured MR data, segmentation of measured MR data, edge contour detection on the basis of measured MR data, for enhancing measured MR data in terms of signal-to-noise ratio by means of noise filtering or apodization, for enhancing measured MR data in terms of resolution by means for deblurring, windowing, zero filling, or generation of gray-scaled images, colour-coded images or images displaying vectors instead of pixels (image data processing or generation, in general G06T)}
- G06T 2207/10072: Tomographic images
- G06T 2207/10116: X-ray image
- G06T 2207/10132: Ultrasound image
- G16H 30/40: for processing medical images, e.g. editing
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G01R: MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES (indicating correct tuning of resonant circuits H03J3/12)
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

A medical image registration method and apparatus are described. By performance of a medical image registration method, a highly accurate registered image, in which breathing deformation information is considered, may be obtained by generation of non-real-time medical images in which the breathing deformation information is reflected before a medical procedure is conducted and by rigid registration of a real-time medical image and the generated non-real-time medical images during the medical procedure.

## Claims

### Claim 1

A medical image registration method comprising:
generating a first medical image set which includes a plurality of first medical images captured during at least two breathing states of an object and a plurality of interpolated medical images of intermediate breathing states that are adjacent to the at least two breathing states;
transmitting ultrasound signals to a region of interest (ROI) of the object and detecting reflected signals corresponding to the transmitted ultrasound signals;
acquiring a second medical image based on the reflected signals;
performing an initial registration, comprising a first rigid registration between at least a first one of the first medical images which is included in the generated first medical image set and the acquired second medical image;
transforming, by applying a transform function, the generated first medical image set by using a registration parameter which is determined based on the initial registration;
down-sampling a plurality of medical images which are included in the transformed first medical image set;
performing a second registration, comprising a second rigid registration, between the down-sampled plurality of medical images and the acquired second medical image;
determining, among the down-sampled plurality of medical images, a down-sampled medical image having a minimum registration cost value among registration cost values obtained by the second registration between the down-sampled plurality of medical images and the acquired second medical image;
updating the down-sampled plurality of medical images to include a plurality of additional interpolated medical images interpolated based on the determined down-sampled medical image;
up-sampling a subset of the updated down-sampled plurality of medical images, the subset including the determined down-sampled medical image;
performing an additional registration during an intraoperative stage, comprising a third rigid registration, between the up-sampled subset of the updated down-sampled plurality of medical images and the second medical image; and
displaying a combined image which is based on a result of the additional registration,
wherein the plurality of interpolated medical images is generated by a first interpolation using a displacement vector which is determined by performing a non-rigid registration with respect to the plurality of first medical images,
wherein the plurality of additional interpolated medical images is generated by a second interpolation using the displacement vector with respect to the determined down-sampled medical image, and
wherein an interval used for segmentation of the displacement vector in the second interpolation is smaller than in the first interpolation.

### Claim 2 (depends on 1)

The medical image registration method of claim 1, wherein the plurality of interpolated medical images are interpolated by performing a linear segmentation or a nonlinear segmentation of the displacement vector with respect to the plurality of first medical images.

### Claim 3 (depends on 1)

The medical image registration method of claim 1, wherein the registration parameter is determined by matching respective landmark points of a first medical image captured during an inhalation state from among the at least two breathing states with corresponding landmark points of the second medical image, and wherein the transforming comprises transforming the first medical images included in the first medical image set by using the determined registration parameter.

### Claim 4 (depends on 1)

The medical image registration method of claim 1, wherein at least three registration parameters are determined by matching respective landmark points of each of at least three from among the first medical images which respectively correspond to an inhalation state, a half inhalation/exhalation state, and an exhalation state from among the at least two breathing states with corresponding landmark points of the second medical image, and wherein an interpolated registration parameter is interpolated using the determined at least three registration parameters, and wherein the transforming comprises using the determined at least three registration parameters and the interpolated registration parameter.

### Claim 5 (depends on 1)

The medical image registration method of claim 1, wherein a plurality of registration parameters are determined by matching respective landmark points of all of the first medical images with corresponding landmark points of the second medical image, and the transforming comprises using all of the plurality of registration parameters.

### Claim 6 (depends on 1)

The medical image registration method of claim 1, wherein the second registration is performed based on an intensity and a gradient of a selected at least first one of the first medical images with respect to the second medical image.

### Claim 7 (depends on 6)

The medical image registration method of claim 6, wherein the second registration is performed for at least three of the first medical images which correspond to an inhalation state, a half inhalation/exhalation state, and an exhalation state from among the at least two breathing states from the transformed first medical image set and the second medical image.

### Claim 8 (depends on 1)

The medical image registration method of claim 1, wherein each of the first medical images includes one from among a magnetic resonance (MR) image, a computed tomography (CT) image, a positron emission tomography (PET) image, and an X-ray image captured before a medical procedure is conducted, and the second medical image includes one from among a 3D ultrasound image and a 2D ultrasound image captured during the medical procedure.

### Claim 9

A medical image registration apparatus comprising:
a memory for executing program data;
an ultrasound probe configured to transmit ultrasound signals to a region of interest (ROI) of an object and detect reflected signals corresponding to the transmitted ultrasound signals; and
at least one processor configured to:
generate a first medical image set which includes a plurality of first medical images captured during at least two breathing states of a subject and a plurality of interpolated medical images of intermediate breathing states that are adjacent to the at least two breathing states;
acquire a second medical image based on the reflected signals;
perform an initial registration, comprising a first rigid registration, between at least a first one of the first medical images which is included in the generated first medical image set and the acquired second medical image;
transform, by applying a transform function, the first medical image set by using a registration parameter which is determined based on the initial registration;
down-sample a plurality of medical images which are included in the transformed first medical image set;
perform a second registration, comprising a second rigid registration, between the down-sampled plurality of medical images and the acquired second medical image;
determine, among the down-sampled plurality of medical images, a down-sampled medical image having a minimum registration cost value among registration cost values obtained by the second registration between the down-sampled plurality of medical images and the acquired second medical image;
update the down-sampled plurality of medical images to include a plurality of additional interpolated medical images interpolated based on the determined down-sampled medical image;
up-sample a subset of the updated down-sampled plurality of medical images, the subset including the determined down-sampled medical image;
perform an additional registration during an intraoperative stage, comprising a third rigid registration, between the up-sampled subset of the updated down-sampled plurality of medical images and the second medical image; and
display a combined image which is based on a result of the additional registration,
wherein the plurality of interpolated medical images is generated by a first interpolation using a displacement vector which is determined by performing a non-rigid registration with respect to the plurality of first medical images,
wherein the plurality of additional interpolated medical images is generated by a second interpolation using the displacement vector with respect to the determined down-sampled medical image, and
wherein an interval used for segmentation of the displacement vector in the second interpolation is smaller than in the first interpolation.

### Claim 10 (depends on 9)

The medical image registration apparatus of claim 9, wherein the plurality of interpolated medical images are interpolated by performing a linear segmentation or a nonlinear segmentation of the displacement vector with respect to the plurality of first medical images.

### Claim 11 (depends on 9)

The medical image registration apparatus of claim 9, wherein the at least one processor is further configured to perform the second registration based on an intensity and a gradient of the at least first one of the first medical images and the second medical image.

### Claim 12 (depends on 9)

The medical image registration apparatus of claim 9, wherein each of the first medical images includes one from among a magnetic resonance (MR) image, a computed tomography (CT) image, a positron emission tomography (PET) image, and an X-ray image captured before a medical procedure is conducted, and the second medical image includes one from among a 3D ultrasound image and a 2D ultrasound image captured during the medical procedure.