US 10,871,536 B2Granted

Automated cardiac volume segmentation

AssigneeArterys Inc.

InventorsDaniel Irving Golden, John Axerio-Cilies, Matthieu Le, Torin Arni Taerum, Jesse Lieman-Sifry

Life of this patent

Estimating the term…

Loading patent…

Abstract

Systems and methods for automated segmentation of anatomical structures, such as the human heart. The systems and methods employ convolutional neural networks (CNNs) to autonomously segment various parts of an anatomical structure represented by image data, such as 3D MRI data. The convolutional neural network utilizes two paths, a contracting path which includes convolution/pooling layers, and an expanding path which includes upsampling/convolution layers. The loss function used to validate the CNN model may specifically account for missing data, which allows for use of a larger training set. The CNN model may utilize multi-dimensional kernels (e.g., 2D, 3D, 4D, 6D), and may include various channels which encode spatial data, time data, flow data, etc. The systems and methods of the present disclosure also utilize CNNs to provide automated detection and display of landmarks in images of anatomical structures.

Claims46

Dependent claims

A system, comprising:

at least one nontransitory processor-readable storage medium that stores at least one of processor-executable instructions or data;
and
at least one processor communicably coupled to the at least one nontransitory processor-readable storage medium, the at least one processor:
receives learning data comprising a plurality of batches of labeled image sets, each image set comprising image data representative of an anatomical structure, and each image set including at least one label which identifies the region of a particular part of the anatomical structure depicted in each image of the image set;
trains a fully convolutional neural network (CNN) model to segment at least one part of the anatomical structure utilizing the received learning data, wherein the CNN model comprises a contracting path and an expanding path, the contracting path includes a number of convolutional layers and a number of pooling layers, each pooling layer preceded by at least one convolutional layer, and the expanding path includes a number of convolutional layers and a number of upsampling layers, each upsampling layer preceded by at least one convolutional layer and comprises a transpose convolution operation which performs upsampling and interpolation with a learned kernel;
stores the CNN model in the at least one nontransitory processor-readable storage medium of the system;
and
processes magnetic resonance imaging (MRI) image data, generated at least in part from an MRI device or system, using the CNN model to generate per-class probabilities for individual pixels of each image of the MRI image data, each class corresponding to one of a plurality of parts of the anatomical structure.

5 more elements

A system, comprising:

at least one nontransitory processor-readable storage medium that stores at least one of processor-executable instructions or data;
and
at least one processor communicably coupled to the at least one nontransitory processor-readable storage medium, the at least one processor:
receives image data generated at least in part from an magnetic resonance imaging (MRI) device or system, wherein the image data represents an anatomical structure;
processes the received image data through a fully convolutional neural network (CNN) model to generate per-class probabilities for each pixel of each image of the image data, each class corresponding to one of a plurality of parts of the anatomical structure represented by the image data, wherein the CNN model comprises a contracting path and an expanding path, the contracting path includes a number of convolutional layers and a number of pooling layers, each pooling layer preceded by at least one convolutional layer, and the expanding path includes a number of convolutional layers and a number of upsampling layers, each upsampling layer preceded by at least one convolutional layer and comprises a transpose convolution operation which performs upsampling and interpolation with a learned kernel;
for each image of the image data, generates a probability map for each of the plurality of classes using the generated per-class probabilities;
and
stores the generated probability maps in the at least one nontransitory processor-readable storage medium.

5 more elements

A method of operating a system comprising at least one nontransitory processor-readable storage medium that stores at least one of processor-executable instructions or data, and at least one processor communicably coupled to the at least one nontransitory processor-readable storage medium, the method comprising:

receiving, by the at least one processor, image data which represents an anatomical structure, the image data including a plurality of magnetic resonance imaging (MRI) images;
processing, by the at least one processor, the received image data through a fully convolutional neural network (CNN) model to generate per-class probabilities for each pixel of each MRI image of the image data, each class corresponding to one of a plurality of parts of the anatomical structure represented by the image data, wherein the CNN model comprises a contracting path and an expanding path, the contracting path includes a number of convolutional layers and a number of pooling layers, each pooling layer preceded by at least one convolutional layer, and the expanding path includes a number of convolutional layers and a number of upsampling layers, each upsampling layer preceded by at least one convolutional layer and comprises a transpose convolution operation which performs upsampling and interpolation with a learned kernel;
for each MRI image of the image data, generating, by the at least one processor, a probability map for each of the plurality of classes using the generated per-class probabilities;
and
storing, by the at least one processor, the generated probability maps in the at least one nontransitory processor-readable storage medium.

2 more elements

receiving image data which represents an anatomical structure, the image data including a plurality of magnetic resonance imaging (MRI) images;
processing, by the at least one processor, the received image data through a fully convolutional neural network (CNN) model to generate per-class probabilities for each pixel of each MRI image of the image data, each class corresponding to one of a plurality of parts of the anatomical structure represented by the image data;
for each MRI image of the image data, generating, by the at least one processor, a probability map for each of the plurality of classes using the generated per-class probabilities;
storing, by the at least one processor, the generated probability maps in the at least one nontransitory processor-readable storage medium;
for each MRI image of the image data, transforming, by the at least one processor, the plurality of probability maps into a label mask by setting the class of each pixel to the class with the highest probability;
and
converting, by the at least one processor, each of the label masks for the image data into respective spline contours.

4 more elements

A system, comprising:

at least one nontransitory processor-readable storage medium that stores at least one of processor-executable instructions or data;
and
at least one processor communicably coupled to the at least one nontransitory processor-readable storage medium, the at least one processor:
receives a plurality of sets of 3D MRI images, the images in each of the plurality of sets represent an anatomical structure of a patient;
receives a plurality of annotations for the plurality of sets of 3D MRI images, each annotation indicative of a landmark of an anatomical structure of a patient depicted in a corresponding image;
trains a convolutional neural network (CNN) model to predict the locations of the plurality of landmarks utilizing the 3D MRI images, wherein the CNN model comprises a contracting path and an expanding path, the contracting path includes a number of convolutional layers and a number of pooling layers, each pooling layer preceded by one or more convolutional layers, and the expanding path includes a number of convolutional layers and a number of upsampling layers, each upsampling layer preceded by one or more convolutional layers and comprises a transpose convolution operation which performs upsampling and interpolation with a learned kernel;
stores the CNN model in the at least one nontransitory processor-readable storage medium of the system;
and
processes MRI image data using the CNN model to generate per-class probabilities for individual pixels of each image of the MRI image data, each class corresponding to one of a plurality of parts of the anatomical structure.

6 more elements

Dependencies are inferred from the claim text.

Facts

Application: 15/779,448Filed 29 Nov 2016 · us provisional application
Priority: 29 Nov 2015
Published: US 2018/0259608 A113 Sep 2018
Granted: 22 Dec 2020
Term ends: Not estimatedThe source records don’t support an estimate.

Classification23 codes

G01R 33/5608
Image enhancement or correction, e.g. subtraction or averaging techniques , e.g. improvement of signal-to-noise ratio and resolution: data…
G01R 33/56308
Of moving material, e.g. flow contrast angiography: characterization of motion or flow; Dynamic imaging
G06N 3/02
Computing arrangements based on biological models: neural networks
G06N 3/045
Architecture, e.g. interconnection topology: combinations of networks