# US 2019/0244357 A1 · System for Segmentation of Anatomical Structures in Cardiac CTA Using Fully Convolutional Neural Networks

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0244357 A1
- Document ID: US20190244357A1
- Status: Published application
- Kind: A1
- Application number: 15/963,442
- Filed: 2018-04-26
- Published: 2019-08-08
- Earliest claimed: 2018-02-07
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Ahmed El Harouni; Mehdi Moradi; Prasanth Prasanna; Tanveer F. Syeda-Mahmood; Hui Tang; Gopalkrishna Veni; Hongzhi Wang
- Web page: https://patentgrep.com/patent/US20190244357A1

## CPC

- G06T 7/11: Region-based segmentation
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06T 7/0012: {Biomedical image inspection}
- G06T 7/194: involving foreground-background segmentation
- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06V 10/82: using neural networks
- G06V 30/19173: {Classification techniques}
- G16H 30/20: for handling medical images, e.g. DICOM, HL7 or PACS
- G16H 30/40: for processing medical images, e.g. editing
- G16H 50/70: for mining of medical data, e.g. analysing previous cases of other patients
- G06T 2207/10072: Tomographic images
- G06T 2207/10081: Computed x-ray tomography [CT]
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30048: Heart; Cardiac
- G06T 2207/30101: Blood vessel; Artery; Vein; Vascular
- G06V 2201/03: Recognition of patterns in medical or anatomical images
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

A method comprises (a) collecting (i) a set of chest computed tomography angiography (CTA) images scanned in the axial view and (ii) a manual segmentation of the images, for each one of multiple organs; (b) preprocessing the images such that they share the same field of view (FOV); (c) using both the images and their manual segmentation to train a supervised deep learning segmentation network, wherein loss is determined from a multi-dice score that is the summation of the dice scores for all the multiple organs, each dice score being computed as the similarity between the manual segmentation and the output of the network for one of the organs; (d) testing a given (input) pre-processed image on the trained network, thereby obtaining segmented output of the given image; and (e) smoothing the segmented output of the given image.

## Claims

### Claim 1

A method, in a data processing system comprising a processor and a memory, the memory comprising instructions that are executed by the processor to specifically configure the processor to implement a system for segmentation of anatomical structures in cardiac computed tomography angiography (CTA) using fully convolutional neural networks, the method comprising:
(a) collecting (i) a set of chest CTA images scanned in the axial view and (ii) a manual segmentation of the images, for each one of multiple anatomical structures;
(b) pre-processing the images such that they share the same field of view (FOV);
(c) using both the images and their manual segmentation to train a supervised deep learning segmentation network, wherein loss is determined from a multi-dice score that is a summation of dice scores for all the multiple anatomical structures, each dice score being computed as the similarity between the manual segmentation and the output of the network for one of the anatomical structures;
(d) testing a given pre-processed image on the trained network, thereby obtaining segmented output of the given image; and
(e) performing post-processing on the segmented output of the given image.

### Claim 2 (depends on 1)

The method of claim 1, wherein pre-processing the images comprises down-sampling the segmented output of the given image.

### Claim 3 (depends on 2)

The method of claim 2, wherein performing post-processing comprises performing up-sampling on the segmented output of the given image.

### Claim 4 (depends on 2)

The method of claim 2, wherein performing post-processing comprises performing smoothing on the segmented output of the given image.

### Claim 5 (depends on 1)

The method of claim 1, wherein the supervised deep learning segmentation network is extended to a multi-label network, where the goal is to find segmentation mask image F, for anatomical structure i, by establishing that fji ∈ Fi, where fji is the jth voxel of the ith segmented foreground, given the ground truth that gji is in the ith ground truth structure, and wherein the overall dice score between the segmentation and the ground truth can be differentiated w.r.t the ithsegmentation's jth voxel, yielding the gradient:
δ


D
δ


f
j
i
=
2

(
g
j
i

(
∑
n
N

f
n
i
2
+
∑
n
N

g
n
i
2
)
-
2

f
j
i

∑
n
N

f
n
i

g
n
i
(
∑
n
N

f
n
i
2
+
∑
n
N

g
n
i
2
)
2
)
where N is the total number of voxels in the image.

### Claim 6 (depends on 1)

The method of claim 1, wherein the segmented output of the given image is segmented for the multiple anatomical structures.

### Claim 7 (depends on 1)

The method of claim 1, wherein multiple anatomical structures include ascending/descending/arch/root aorta, left/right/trunk pulmonary artery, vertebrae, left/right atrium, left/right ventricle, left ventricular myocardium, and superior/inferior vena cava.

### Claim 8 (depends on 1)

The method of claim 1, wherein pre-processing the images comprises intensity equalizing the images to eliminate intensity variation between patients.

### Claim 9

A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a computing device, causes the computing device to implement a system for segmentation of anatomical structures in cardiac computed tomography angiography (CTA) using fully convolutional neural networks, wherein the computer readable program causes the computing device to:
(a) collect (i) a set of chest CTA images scanned in the axial view and (ii) a manual segmentation of the images, for each one of multiple anatomical structures;
(b) pre-process the images such that they share the same field of view (FOV);
(c) use both the images and their manual segmentation to train a supervised deep learning segmentation network, wherein loss is determined from a multi-dice score that is a summation of dice scores for all the multiple anatomical structures, each dice score being computed as the similarity between the manual segmentation and the output of the network for one of the anatomical structures;
(d) test a given pre-processed image on the trained network, thereby obtaining segmented output of the given image; and
(e) perform post-processing on the segmented output of the given image.

### Claim 10 (depends on 9)

The computer program product of claim 9, wherein pre-processing the images comprises down-sampling the segmented output of the given image.

### Claim 11 (depends on 10)

The computer program product of claim 10, wherein performing post-processing comprises performing up-sampling on the segmented output of the given image.

### Claim 12 (depends on 10)

The computer program product of claim 10, wherein performing post-processing comprises performing smoothing on the segmented output of the given image.

### Claim 13 (depends on 9)

The computer program product of claim 9, wherein the supervised deep learning segmentation network is extended to a multi-label network, where the goal is to find segmentation mask image F, for anatomical structure i, by establishing that fji ∈Fi, where fji is the jth voxel of the ithsegmented foreground, given the ground truth that gji is in the ith ground truth structure, and wherein the overall dice score between the segmentation and the ground truth can be differentiated w.r.t the ith segmentation's jth voxel, yielding the gradient:
δ


D
δ


f
j
i
=
2

(
g
j
i

(
∑
n
N

f
n
i
2
+
∑
n
N

g
n
i
2
)
-
2

f
j
i

∑
n
N

f
n
i

g
n
i
(
∑
n
N

f
n
i
2
+
∑
n
N

g
n
i
2
)
2
)
where N is the total number of voxels in the image.

### Claim 14 (depends on 9)

The computer program product of claim 9, wherein the segmented output of the given image is segmented for the multiple anatomical structures.

### Claim 15 (depends on 9)

The computer program product of claim 9, wherein multiple anatomical structures include ascending/descending/arch/root aorta, left/right/trunk pulmonary artery, vertebrae, left/right atrium, left/right ventricle, left ventricular myocardium, and superior/inferior vena cava.

### Claim 16 (depends on 9)

The computer program product of claim 9, wherein pre-processing the images comprises intensity equalizing the images to eliminate intensity variation between patients.

### Claim 17

An apparatus comprising:
at least one processor; and
a memory coupled to the at least one processor, wherein the memory comprises instructions which, when executed by the at least one processor, cause the at least one processor to implement a system for segmentation of anatomical structures in cardiac computed tomography angiography (CTA) using fully convolutional neural networks, wherein the instructions cause the at least one processor to:
(a) collect (i) a set of chest CTA images scanned in the axial view and (ii) a manual segmentation of the images, for each one of multiple anatomical structures;
(b) pre-process the images such that they share the same field of view (FOV);
(c) use both the images and their manual segmentation to train a supervised deep learning segmentation network, wherein loss is determined from a multi-dice score that is a summation of dice scores for all the multiple anatomical structures, each dice score being computed as the similarity between the manual segmentation and the output of the network for one of the anatomical structures;
(d) test a given pre-processed image on the trained network, thereby obtaining segmented output of the given image; and
(e) perform post-processing on the segmented output of the given image.

### Claim 18 (depends on 17)

The apparatus of claim 17, wherein pre-processing the images comprises down-sampling the segmented output of the given image and wherein performing post-processing comprises performing up-sampling on the segmented output of the given image.

### Claim 19 (depends on 18)

The apparatus of claim 18, wherein performing post-processing comprises performing smoothing on the segmented output of the given image.

### Claim 20 (depends on 17)

The apparatus of claim 17, wherein the supervised deep learning segmentation network is extended to a multi-label network, where the goal is to find segmentation mask image Fi for anatomical structure i, by establishing that fji ∈Fti, where fji is the jth voxel of the ith segmented foreground, given the ground truth that gji is in the ith ground truth structure, and wherein the overall dice score between the segmentation and the ground truth can be differentiated w.r.t the ithsegmentation's jth voxel, yielding the gradient:
δ


D
δ


f
j
i
=
2

(
g
j
i

(
∑
n
N

f
n
i
2
+
∑
n
N

g
n
i
2
)
-
2

f
j
i

∑
n
N

f
n
i

g
n
i
(
∑
n
N

f
n
i
2
+
∑
n
N

g
n
i
2
)
2
)
where N is the total number of voxels in the image.
