# US 2016/0260210 A1 · Method and system for determining the prognosis of a patient suffering from pulmonary embolism

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0260210 A1
- Document ID: US20160260210A1
- Status: Published application
- Kind: A1
- Application number: 15/026,401
- Filed: 2014-11-27
- Published: 2016-09-08
- Earliest claimed: 2013-11-27
- Assignees: Universidad Politecnica De Madrid; Massachusetts Institute of Technology; The Brigham and Women's Hospital, Inc.
- Applicants: Universidad PolitéCnica De Madrid; Massachusetts Institute of Technology; Brigham and Women's Hospital
- Inventors: Germán SERRANO GONZÁLEZ; Daniel CARRETERO JIMÉNEZ; Frank JOHN RYBICKI; María J. LEDESMA CARBAYO; Sara LÓPEZ RODRÍGUEZ; Raúl ESTÉPAR SAN JOSÉ
- Web page: https://patentgrep.com/patent/US20160260210A1

## CPC

- G06T 7/0012: {Biomedical image inspection}
- A61B 5/7271: {Specific aspects of physiological measurement analysis (specific diagnostics methods using bioelectric or biomagnetic signals A61B5/316)}
- G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques}
- G06F 18/22: Matching criteria, e.g. proximity measures
- G06F 18/23: Clustering techniques
- G06F 18/2415: based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
- G06T 7/13: Edge detection
- G06T 7/60: Analysis of geometric attributes
- G06T 7/62: of area, perimeter, diameter or volume
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 7/75: {involving models}
- G06T 15/08: Volume rendering
- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- G06V 30/1916: {Validation; Performance evaluation}
- G06V 30/19173: {Classification techniques}
- G06T 2200/04: involving 3D image data
- G06T 2207/10081: Computed x-ray tomography [CT]
- G06T 2207/20081: Training; Learning
- G06T 2207/20161: Level set
- G06T 2207/30048: Heart; Cardiac
- G06V 2201/031: of internal organs
- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- G16H 50/50: for simulation or modelling of medical disorders
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

A system for determining the prognosis of a patient suffering from pulmonary embolism is provided. The system may include at least one computer system configure to receive patient specific data regarding his pulmonary embolism status. The at least one computer system may be further configured to create a model of the patient's heart, with at least information of the two ventricles, and to determine the ratio of sizes of the ventricles. The system will then report such ratio to the clinician or report a risk index of clinical outcome for such patient.

## Claims

### Claim 1

A method for computing the Right Ventricle:Left Ventricle diameter ratio in a subject, the method comprising:
receiving a plurality of axial images of the patient showing at least the heart, forming a three-dimensional image formed by voxels;
automatically estimating, via a computer system, in at least one three-dimensional image the location and shape of the contour of the ventricles of the heart of the patient;
automatically estimating, via a computer system, in at least one three-dimensional image the location of the inter-ventricular septum of the heart of the patient modeled as a plane;
automatically estimating, via a computer system, the diameter of the ventricles of the heart as the maximum width of the contour of each ventricle measured in lines perpendicular to the interventricular septum;
reporting the value of the division of the maximum measurement for a right ventricle to a maximum measurement for a left ventricle.

### Claim 2 (depends on 1)

The method according to claim 1, wherein automatically estimating in the image the location and shape of the contour of the ventricles further comprises:
providing an axial slice formed as a 2-dimensional model of the shape of the right ventricle and an axial slice as a 2-dimensional model of the shape of the left ventricle by training a machine learning based algorithm;
detecting in each axial slice the shape and location of the right ventricle and the shape and location of the left ventricle by said trained algorithm; and
for each detected ventricle in each axial slice, providing a 2-dimensional bounding box and a score representing the fitness of the detection to each model.

### Claim 3 (depends on 2)

The method according to claim 2, further comprising:
selecting detected ventricles that are coherent with size and locations of 2-dimensional models previously recorded when modeling;
clustering detected ventricles according to the similarities on position of the detections, their sizes and their aspect ratio;
ranking clusters and selecting the cluster with the highest score for the left and the right ventricle; and
for the left and for the right ventricles of the selected cluster, locating a seed point in the axial image related to the centroid of said selected ventricle.

### Claim 4 (depends on 3)

The method according to claim 3, wherein detected ventricles are coherent with size and locations of models when:
absolute value of the difference between the area of the detected ventricle and the model is less than a predetermined predefined value; and
absolute value of the difference between the position of the mass centre of the image and the centroid of the detected ventricle is less than a predefined value.

### Claim 5 (depends on 1)

The method according to claim 1, wherein automatically estimating the location of the inter-ventricular septum of the heart modeled as a plane comprises:
for each pair of seeds, the seed of the right ventricle and the seed of the left ventricle, computing the line connecting the pair of seeds;
for each line, calculating the location of the voxel wherein the three-dimensional image intensity is a maximum if the intensity of the septum is higher than the intensity of the ventricle, or a minimum if the intensity of the septum is lower than the intensity of the ventricle;
defining a volume comprised by the voxels previously calculated;
computing the plane that represent the inter-ventricular septum form from the plurality of voxels of such volume, wherein said computing comprises:
for each voxel, computing the hessian matrix of the three-dimensional image at such voxel location being the hessian matrix
(
H
xx
H
xy
H
xz
H
yx
H
yy
H
yz
H
zx
H
zy
H
zz
)
wherein Hij, i,j=x, y, z represents the convolution of the image with the second derivative along the dimensions i,j of a gaussian kernel of a predetermined sigma;
for each voxel, computing the eigenvalues and eigenvectors of the associated hessian matrix;
for each voxel determining a score proportional to a plate-like structure traversing said voxel;
selecting a predetermined number of voxels within the volume with the highest score;
calculating the median direction of the eigenvectors of the selected voxels, said median direction being the direction of the estimated septum; and
defining the plane of the septum as the plane containing the voxel whose eigenvector is closest in direction to the median direction and the normal of the plane being the median direction.

### Claim 6 (depends on 1)

The method according to claim 1, wherein each ventricle is automatically segmented, via a computer system, by evolving a contour form from at least one seed point by means of a level-set algorithm under constraints over the three-dimensional image.

### Claim 7 (depends on 6)

The method according to claim 6, wherein the three-dimensional image constraints are defined by a learning pre-process wherein the level-set algorithm is run over a plurality of cases taken from an evaluation dataset.

### Claim 8 (depends on 1)

The method according to claim 1, wherein automatically estimating, via a computer system, the diameter of the ventricles further comprises:
for each axial image, determining the septum line as the intersection between the plane of the septum and the axial image;
defining the cross measurement of the ventricle as the distance value between the closest point of the contour and the farthest point of the contour along a line perpendicular to the septum line at certain points xi along the septum line from the apex to the atrium;
defining a discrete function pi=f(xi), i=1 . . . n for a predetermined number of sample points n xi have cross measurements pi;
fitting the discrete function pi=f(xi), i=1 . . . n by a polynomial of order n; and
determining the estimated diameter or caliper as the value of the polynomial at the at least one root of the first derivative of the polynomial.

### Claim 9 (depends on 8)

The method according to claim 8, wherein the estimation of the right ventricle diameter further comprises:
defining a closed domain of the polynomial as the domain where the cross measurement exists;
calculating the first derivative pn-1′(x) from pn(x) and the roots of the first derivative (rpd);
calculating the second derivative pn-2″(x) from pn(x) and the second derivative's roots (rpdd);
selecting the root of the polynomial for the estimation of the diameter of the right ventricle according to the following criteria:
in the event that there are three or more roots in pn-1′(x), and pn-2″(x) on the first root, being the roots sorted from the left boundary to the right boundary of the closed domain, is lower than 0, and pn-2″(x) on the second root is greater than 0, and the pn-2″(x) at the third root is lower than 0, taking as root of the polynomial for the estimation of the diameter of the right ventricle the first root of pn-1′(x);
in the event that there are at least two roots in pn-1′(x), and the pn-2″(x) on the first root is lower than 0, and the pn-2″(x) on the second root is greater than 0, taking as root of the polynomial for the estimation of the diameter of the right ventricle the position of the first root of pn-1′(x);
in the event that there is at least one root in pn-2″(x), the value of pn-1′(x) at the first of these roots is greater than 0 and smaller than a predetermined threshold, taking as root of the polynomial for the estimation of the diameter of the right ventricle the value of that root; and
in the event that there is at least one root in pn-1′(x) and pn-2″(x) in the first of the roots is lower than 0, taking as root of the polynomial for the estimation of the diameter of the right ventricle the position of the first root of pn-1′(x).

### Claim 10 (depends on 8)

The method according to claim 8, wherein the estimation of the left ventricle diameter further comprises:
defining a closed domain of the polynomial as the domain where the cross measurement exists;
calculating the first derivative pn-1′(x) from pn(x) and the roots of the first derivative (rpd);
calculating the second derivative pn-2″(x) from pn(x) and the roots of the second derivative (rpdd);
determining whether the value of pn-2″(x) at the first root of pn-1′(x) is less than 0; and
upon determining that the value of pn-2″(x) at the first root of pn-1′(x) is less than 0, taking said first root of pn-1′(x) as root of the polynomial for the estimation of the diameter of the left ventricle.

### Claim 11 (depends on 9)

The method according to claim 9, wherein after calculating the roots of the first derivative pn-1′(x) and the roots of the second derivative pn-2″(x), those roots closer than a predetermined value to the extremes of the domain are removed.

### Claim 12 (depends on 8)

The method according to any of claim 8, wherein the maximum distance value between the closest point of the contour and the farthest point of the contour along a line perpendicular to the septum is estimated as the maximum value of the maximum of the polynomials computed at each axial image.

### Claim 13 (depends on 10)

The method according to claim 10, wherein after calculating the roots of the first derivative pn-1′(x) and the roots of the second derivative pn-2″(x), those roots closer than a predetermined value to the extremes of the domain are removed.

### Claim 14

A computer-readable medium storing code, which when executed by a processor, causes the processor:
to receive a plurality of axial images of the patient showing at least the heart, forming a three-dimensional image formed by voxels;
to automatically estimating, via a computer system, in at least one three-dimensional image, the location and shape of the contour of the ventricles of the heart of the patient;
to automatically estimating, via a computer system, in at least one three-dimensional image, the location of the inter-ventricular septum of the heart of the patient modeled as a plane;
to automatically estimating, via a computer system, the diameter of the ventricles of the heart as the maximum width of the contour of each ventricle measured in lines perpendicular to the interventricular septum; and
to report the value of the division of the maximum measurement for a right ventricle to a maximum measurement for a left ventricle.

### Claim 15

An electronic device for data analysis, the device comprising
a processor; and
a computer-readable medium storing computer-executable instructions which, when executed by the processor, causes the processor:
to receive a plurality of axial images of the patient showing at least the heart, forming a three-dimensional image formed by voxels;
to automatically estimating, via a computer system, in at least one three-dimensional image, the location and shape of the contour of the ventricles of the heart of the patient;
to automatically estimating, via a computer system, in at least one three-dimensional image, the location of the inter-ventricular septum of the heart of the patient modeled as a plane;
to automatically estimating, via a computer system, the diameter of the ventricles of the heart as the maximum width of the contour of each ventricle measured in lines perpendicular to the interventricular septum; and
to report the value of the division of the maximum measurement for a right ventricle to a maximum measurement for a left ventricle.

### Claim 16 (depends on 14)

The computer-readable medium of claim 14, wherein the code, when executed by the processor, further causes the processor:
to provide an axial slice formed as a 2-dimensional model of the shape of the right ventricle and an axial slice as a 2-dimensional model of the shape of the left ventricle by training a machine learning based algorithm;
to detect in each axial slice the shape and location of the right ventricle and the shape and location of the left ventricle by said trained algorithm; and
for each detected ventricle in each axial slice, to provide a 2-dimensional bounding box and a score representing the fitness of the detection to each model.

### Claim 17 (depends on 14)

The computer-readable medium of claim 14, wherein the code, when executed by the processor, further causes the processor:
for each pair of seeds, the seed of the right ventricle and the seed of the left ventricle, to compute the line connecting the pair of seeds;
for each line, to calculate the location of the voxel wherein the three-dimensional image intensity is a maximum if the intensity of the septum is higher than the intensity of the ventricle, or a minimum if the intensity of the septum is lower than the intensity of the ventricle;
to define a volume comprised by the voxels previously calculated;
to compute the plane that represent the inter-ventricular septum form from the plurality of voxels of such volume, wherein said computing comprises:
for each voxel, computing the hessian matrix of the three-dimensional image at such voxel location being the hessian matrix
(
H
xx
H
xy
H
xz
H
yx
H
yy
H
yz
H
zx
H
zy
H
zz
)
wherein Hij, i,j=x, y, z represents the convolution of the image with the second derivative along the dimensions i,j of a gaussian kernel of a predetermined sigma;
for each voxel, computing the eigenvalues and eigenvectors of the associated hessian matrix;
for each voxel determining a score proportional to a plate-like structure traversing said voxel;
selecting a predetermined number of voxels within the volume with the highest score;
calculating the median direction of the eigenvectors of the selected voxels, said median direction being the direction of the estimated septum; and
defining the plane of the septum as the plane containing the voxel whose eigenvector is closest in direction to the median direction and the normal of the plane being the median direction.

### Claim 18 (depends on 14)

The computer-readable medium of claim 14, wherein the code, when executed by the processor, further causes the processor:
for each axial image, to determine the septum line as the intersection between the plane of the septum and the axial image;
to define the cross measurement of the ventricle as the distance value between the closest point of the contour and the farthest point of the contour along a line perpendicular to the septum line at certain points xi along the septum line from the apex to the atrium;
to define a discrete function pi=f(xi), i=1 . . . n for a predetermined number of sample points n wherein xi have cross measurements, pi;
to fit the discrete function pi=f(xi), i=1 . . . n by a polynomial of order n; and
to determine the estimated diameter or caliper as the value of the polynomial at the at least one root of the first derivative of the polynomial.

### Claim 19 (depends on 15)

The electronic device of claim 15, wherein the computer-executable instructions, when executed by the processor further causes the processor:
to provide an axial slice formed as a 2-dimensional model of the shape of the right ventricle and an axial slice as a 2-dimensional model of the shape of the left ventricle by training a machine learning based algorithm;
to detect in each axial slice the shape and location of the right ventricle and the shape and location of the left ventricle by said trained algorithm; and
for each detected ventricle in each axial slice, to provide a 2-dimensional bounding box and a score representing the fitness of the detection to each model.

### Claim 20 (depends on 15)

The electronic device of claim 15, wherein the computer-executable instructions, when executed by the processor further causes the processor:
for each pair of seeds, the seed of the right ventricle and the seed of the left ventricle, to compute the line connecting the pair of seeds;
for each line, to calculate the location of the voxel wherein the three-dimensional image intensity is a maximum if the intensity of the septum is higher than the intensity of the ventricle, or a minimum if the intensity of the septum is lower than the intensity of the ventricle;
to define a volume comprised by the voxels previously calculated;
to compute the plane that represent the inter-ventricular septum form from the plurality of voxels of such volume, wherein said computing comprises:
for each voxel, computing the hessian matrix of the three-dimensional image at such voxel location being the hessian matrix
(
H
xx
H
xy
H
xz
H
yx
H
yy
H
yz
H
zx
H
zy
H
zz
)
wherein Hij, i,j=x, y, z represents the convolution of the image with the second derivative along the dimensions i,j of a gaussian kernel of a predetermined sigma;
for each voxel, computing the eigenvalues and eigenvectors of the associated hessian matrix;
for each voxel determining a score proportional to a plate-like structure traversing said voxel;
selecting a predetermined number of voxels within the volume with the highest score;
calculating the median direction of the eigenvectors of the selected voxels, said median direction being the direction of the estimated septum; and
defining the plane of the septum as the plane containing the voxel whose eigenvector is closest in direction to the median direction and the normal of the plane being the median direction.
