US 10,896,508 B2Granted

System for segmentation of anatomical structures in cardiac CTA using fully convolutional neural networks

AssigneeInternational Business Machines Corporation

InventorsAhmed El Harouni, Mehdi Moradi, Prasanth Prasanna, Tanveer F. Syeda-Mahmood, Hui Tang, Gopalkrishna Veni, Hongzhi Wang

Life of this patent

Estimating the term…

patentgrep

Loading patent…

Abstract

A method comprises (a) collecting (i) a set of chest computed tomography angiography (CTA) images scanned in the axial view and (ii) a manual segmentation of the images, for each one of multiple organs; (b) preprocessing the images such that they share the same field of view (FOV); (c) using both the images and their manual segmentation to train a supervised deep learning segmentation network, wherein loss is determined from a multi-dice score that is the summation of the dice scores for all the multiple organs, each dice score being computed as the similarity between the manual segmentation and the output of the network for one of the organs; (d) testing a given (input) pre-processed image on the trained network, thereby obtaining segmented output of the given image; and (e) smoothing the segmented output of the given image.

Claims17

Dependent claims

A method, in a data processing system comprising a processor and a memory, the memory comprising instructions that are executed by the processor to specifically configure the processor to implement a system for segmentation of anatomical structures in cardiac computed tomography angiography (CTA) using fully convolutional neural networks, the method comprising:

(a) collecting (i) a set of chest CTA images scanned in the axial view and (ii) a manual segmentation of the images, for each one of multiple anatomical structures, wherein the multiple anatomical structures include ascending/descending/arch/root aorta, left/right/trunk pulmonary artery, vertebrae, left/right atrium, left/right ventricle, left ventricular myocardium, and superior/inferior vena cava;
(b) pre-processing the images such that they share the same field of view (FOV);
(c) using both the images and their manual segmentation to train a supervised deep learning segmentation network, wherein loss is determined from a multi-dice score that is a summation of dice scores for all the multiple anatomical structures, each dice score being computed as the similarity between the manual segmentation and the output of the network for one of the anatomical structures;
(d) testing a given pre-processed image on the trained network, thereby obtaining segmented output of the given image, wherein the segmented output of the given image is segmented for the multiple anatomical structures;
and
(e) performing post-processing on the segmented output of the given image.

3 more elements

A method in a data processing system comprising a processor and a memory, the memory comprising instructions that are executed by the processor to specifically configure the processor to implement a system for segmentation of anatomical structures in cardiac computed tomography angiography (CTA) using fully convolutional neural networks, the method comprising:

(a) collecting (i) a set of chest CTA images scanned in the axial view and (ii) a manual segmentation of the images, for each one of multiple anatomical structures;
(b) pre-processing the images such that they share the same field of view (FOV);
(c) using both the images and their manual segmentation to train a supervised deep learning segmentation network, wherein loss is determined from a multi-dice score that is a summation of dice scores for all the multiple anatomical structures, each dice score being computed as the similarity between the manual segmentation and the output of the network for one of the anatomical structures, wherein the supervised deep learning segmentation network is extended to a multi-label network, where the goal is to find find segmentation mask image Fi, for anatomical structure i, by establishing that fji ∈ Fi, where fji is the jth voxel of the ith segmented foreground, given the ground truth that gij is in the ith ground truth structure, and wherein the overall dice score between the segmentation and the ground truth can be differentiated w.r.t the ith segmentation's jth voxel, yielding the gradient:
δ
⁢
⁢
D
δ
⁢
⁢
f
j
i
=
2
⁢
(
g
j
i
⁡
(
∑
n
N
⁢
f
n
i
2
+
∑
n
N
⁢
g
n
i
2
)
-
2
⁢
f
j
i
⁢
∑
n
N
⁢
f
n
i
⁢
g
n
i
(
∑
n
N
⁢
f
n
i
2
+
∑
n
N
⁢
g
n
i
2
)
2
)
where N is the total number of voxels in the image;
(d) testing a given pre-processed image on the trained network, thereby obtaining segmented output of the given image, wherein the segmented output of the given image is segmented for the multiple anatomical structures;
and
(e) performing post-processing on the segmented output of the given image.

80 more elements

A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a computing device, causes the computing device to implement a system for segmentation of anatomical structures in cardiac computed tomography angiography (CTA) using fully convolutional neural networks, wherein the computer readable program causes the computing device to:

(a) collect (i) a set of chest CTA images scanned in the axial view and (ii) a manual segmentation of the images, for each one of multiple anatomical structures, wherein the multiple anatomical structures include ascending/descending/arch/root aorta, left/right/trunk pulmonary artery, vertebrae, left/right atrium, left/right ventricle, left ventricular myocardium, and superior/inferior vena cava;
(b) pre-process the images such that they share the same field of view (FOV);
(c) use both the images and their manual segmentation to train a supervised deep learning segmentation network, wherein loss is determined from a multi-dice score that is a summation of dice scores for all the multiple anatomical structures, each dice score being computed as the similarity between the manual segmentation and the output of the network for one of the anatomical structures;
(d) test a given pre-processed image on the trained network, thereby obtaining segmented output of the given image, wherein the segmented output of the given image is segmented for the multiple anatomical structures;
and
(e) perform post-processing on the segmented output of the given image.

3 more elements

(a) collect (i) a set of chest CTA images scanned in the axial view and (ii) a manual segmentation of the images, for each one of multiple anatomical structures;
(b) pre-process the images such that they share the same field of view (FOV);
(c) use both the images and their manual segmentation to train a supervised deep learning segmentation network, wherein loss is determined from a multi-dice score that is a summation of dice scores for all the multiple anatomical structures, each dice score being computed as the similarity between the manual segmentation and the output of the network for one of the anatomical structures, wherein the supervised deep learning segmentation network is extended to a multi-label network, where the goal is to find segmentation mask image Fi, for anatomical structure i, by establishing that fji ∈ Fi, where fji is the jth voxel of the ith segmented foreground, given the ground truth that gij is in the ith ground truth structure, and wherein the overall dice score between the segmentation and the ground truth can be differentiated w.r.t the ith segmentation's jth voxel, yielding the gradient:
δ
⁢
⁢
D
δ
⁢
⁢
f
j
i
=
2
⁢
(
g
j
i
⁡
(
∑
n
N
⁢
f
n
i
2
+
∑
n
N
⁢
g
n
i
2
)
-
2
⁢
f
j
i
⁢
∑
n
N
⁢
f
n
i
⁢
g
n
i
(
∑
n
N
⁢
f
n
i
2
+
∑
n
N
⁢
g
n
i
2
)
2
)
where N is the total number of voxels in the image;
(d) test a given pre-processed image on the trained network, thereby obtaining segmented output of the given image, wherein the segmented output of the given image is segmented for the multiple anatomical structures;
and
(e) perform post-processing on the segmented output of the given image.

80 more elements

An apparatus comprising:

at least one processor;
and
a memory coupled to the at least one processor, wherein the memory comprises instructions which, when executed by the at least one processor, cause the at least one processor to implement a system for segmentation of anatomical structures in cardiac computed tomography angiography (CTA) using fully convolutional neural networks, wherein the instructions cause the at least one processor to:
(a) collect (i) a set of chest CTA images scanned in the axial view and (ii) a manual segmentation of the images, for each one of multiple anatomical structures, wherein the multiple anatomical structures include ascending/descending/arch/root aorta, left/right/trunk pulmonary artery, vertebrae, left/right atrium, left/right ventricle, left ventricular myocardium, and superior/inferior vena cava;
(b) pre-process the images such that they share the same field of view (FOV);
(c) use both the images and their manual segmentation to train a supervised deep learning segmentation network, wherein loss is determined from a multi-dice score that is a summation of dice scores for all the multiple anatomical structures, each dice score being computed as the similarity between the manual segmentation and the output of the network for one of the anatomical structures;
(d) test a given pre-processed image on the trained network, thereby obtaining segmented output of the given image, wherein the segmented output of the given image is segmented for the multiple anatomical structures;
and
(e) perform post-processing on the segmented output of the given image.

6 more elements

An apparatus comprising:

at least one processor;
and
a memory coupled to the at least one processor, wherein the memory comprises instructions which, when executed by the at least one processor, cause the at least one processor to implement a system for segmentation of anatomical structures in cardiac computed tomography angiography (CTA) using fully convolutional neural networks, wherein the instructions cause the at least one processor to:
(a) collect (i) a set of chest CTA images scanned in the axial view and (ii) a manual segmentation of the images, for each one of multiple anatomical structures;
(b) pre-process the images such that they share the same field of view (FOV);
(c) use both the images and their manual segmentation to train a supervised deep learning segmentation network, wherein loss is determined from a multi-dice score that is a summation of dice scores for all the multiple anatomical structures, each dice score being computed as the similarity between the manual segmentation and the output of the network for one of the anatomical structures, wherein the supervised deep learning segmentation network is extended to a multi-label network, where the goal is to find find segmentation mask image Fi, for anatomical structure i, by establishing that fji ∈ Fi, where fji is the jth voxel of the ith segmented foreground, given the ground truth that gij is in the ith ground truth structure, and wherein the overall dice score between the segmentation and the ground truth can be differentiated w.r.t the ith segmentation's jth voxel, yielding the gradient:
δ
⁢
⁢
D
δ
⁢
⁢
f
j
i
=
2
⁢
(
g
j
i
⁡
(
∑
n
N
⁢
f
n
i
2
+
∑
n
N
⁢
g
n
i
2
)
-
2
⁢
f
j
i
⁢
∑
n
N
⁢
f
n
i
⁢
g
n
i
(
∑
n
N
⁢
f
n
i
2
+
∑
n
N
⁢
g
n
i
2
)
2
)
where N is the total number of voxels in the image;
(d) test a given pre-processed image on the trained network, thereby obtaining segmented output of the given image, wherein the segmented output of the given image is segmented for the multiple anatomical structures;
and
(e) perform post-processing on the segmented output of the given image.

83 more elements

Dependencies are inferred from the claim text.

Facts

Application: 15/963,442Filed 26 Apr 2018 · us provisional application
Priority: 7 Feb 2018
Published: US 2019/0244357 A18 Aug 2019
Granted: 19 Jan 2021
Term ends: Not estimatedThe source records don’t support an estimate.

Classification24 codes

G06T 7/11
Segmentation; Edge detection: region-based segmentation
G06N 3/045
Architecture, e.g. interconnection topology: combinations of networks
G06N 3/0464
Architecture, e.g. interconnection topology: convolutional networks [CNN, ConvNet]
G06N 3/08
Neural networks: learning methods