# US 10,441,215 B2 · Predicting immunotherapy response in non-small cell lung cancer with serial quantitative vessel tortuosity

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,441,215 B2
- Document ID: US10441215B2
- Status: Granted
- Kind: B2
- Application number: 15/893,086
- Filed: 2018-02-09
- Published: 2018-08-30
- Granted: 2019-10-15
- Earliest claimed: 2017-02-27
- Assignees: Case Western Reserve University
- Applicants: Case Western Reserve University
- Inventors: Anant Madabhushi; Yuanqi Xie; Vamsidhar Velcheti
- Web page: https://patentgrep.com/patent/US10441215B2

## CPC

- A61B 5/7267: {involving training the classification device}
- A61B 5/02007: {Evaluating blood vessel condition, e.g. elasticity, compliance}
- A61B 5/4839: {combined with drug delivery}
- A61B 5/4848: {Monitoring or testing the effects of treatment, e.g. of medication}
- A61B 6/50: specially adapted for specific body parts; specially adapted for specific clinical applications
- A61K 49/0004: {Screening or testing of compounds for diagnosis of disorders, assessment of conditions, e.g. renal clearance, gastric emptying, testing for diabetes, allergy, rheuma, pancreas functions}
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F 18/24: Classification techniques
- G06F 18/2411: based on the proximity to a decision surface, e.g. support vector machines
- G06F 18/2415: based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
- G06F 18/2433: Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
- G06T 7/0014: {using an image reference approach}
- G06T 7/0016: {involving temporal comparison}
- G06T 7/11: Region-based segmentation
- G06T 7/174: involving the use of two or more images
- G06T 7/337: {involving reference images or patches}
- G06V 10/764: using classification, e.g. of video objects
- G16H 30/40: for processing medical images, e.g. editing
- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- G16H 50/30: for calculating health indices; for individual health risk assessment
- G16H 50/50: for simulation or modelling of medical disorders
- A61B 5/4842: {Monitoring progression or stage of a disease}
- A61B 5/7264: {Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems}
- A61B 6/032: {Transmission computed tomography [CT]}
- G06T 2200/04: involving 3D image data
- G06T 2207/10081: Computed x-ray tomography [CT]
- G06T 2207/20076: Probabilistic image processing
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30024: Cell structures in vitro; Tissue sections in vitro
- G06T 2207/30061: Lung
- G06T 2207/30064: Lung nodule
- G06T 2207/30096: Tumor; Lesion
- G06T 2207/30101: Blood vessel; Artery; Vein; Vascular
- G06V 2201/03: Recognition of patterns in medical or anatomical images
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- A61K: PREPARATIONS FOR MEDICAL, DENTAL OR TOILETRY PURPOSES (devices or methods specially adapted for bringing pharmaceutical products into particular physical or administering forms A61J3/00; chemical aspects of, or use of materials for deodorisation of air, for disinfection or sterilisation, or for bandages, dressings, absorbent pads or surgical articles A61L; soap compositions C11D)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

One embodiment includes an image acquisition circuit that accesses a pre-treatment and a post-treatment image of a region of tissue demonstrating non-small cell lung cancer (NSCLC), a segmentation and registration circuit that annotates the tumor represented in the images, and that registers the pre-treatment image with the post-treatment image; a feature extraction circuit that selects a set of pre-treatment and a set of post-treatment quantitative vessel tortuosity (QVT) features from the registered image; a delta-QVT circuit that generates a set of delta-QVT features by computing a difference between the set of post-treatment QVT features and the set of pre-treatment QVT features; and a classification circuit that generates a probability that the region of tissue will respond to immunotherapy based on the difference, and that classifies the region of tissue as a responder or non-responder. Embodiments may generate an immunotherapy treatment plan based on the classification.

## Claims

### Claim 1

An apparatus for predicting patient response to immunotherapy, the apparatus comprising:
a processor;
a memory that stores a digitized computed tomography (CT) pre-treatment image and at least one digitized CT post-treatment image of a region of tissue demonstrating non-small cell lung cancer (NSCLC), where the region of tissue includes a tumor, where the pre-treatment image includes a plurality of voxels and the at least one digitized CT post-treatment image includes a plurality of voxels;
an input/output (I/O) interface;
a set of circuits comprising an image acquisition circuit, a segmentation circuit, a registration circuit, a quantitative vessel tortuosity (QVT) feature extraction circuit, a delta-QVT circuit, and a classification circuit; and
an interface to connect the processor, the memory, the I/O interface and the set of circuits;
the image acquisition circuit configured to access the pre-treatment image and the at least one digitized CT post-treatment image;
the segmentation circuit configured to:
annotate the tumor represented in the pre-treatment image;
annotate a three dimensional (3D) segmented vasculature associated with the tumor represented in the pre-treatment image;
identify a center line of the 3D segmented vasculature associated with the tumor represented in the pre-treatment image;
annotate the tumor represented in the at least one digitized CT post-treatment image;
annotate a 3D segmented vasculature associated with the tumor represented in the at least one digitized CT post-treatment image; and
identify a center line of the 3D segmented vasculature associated with the tumor represented in the post-treatment image;
the registration circuit configured to generate a registered image by registering the pre-treatment image with the at least one digitized CT post-treatment image;
the QVT feature extraction circuit configured to:
select a set of pre-treatment QVT features from the registered image; and
select a set of post-treatment QVT features from the registered image;
the delta-QVT circuit configured to:
generate a set of delta-QVT features by computing a difference between the set of post-treatment QVT features and the set of pre-treatment QVT features; and
provide the set of delta-QVT features to the classification circuit; and
the classification circuit configured to:
generate a probability that the region of tissue will respond to immunotherapy based, at least in part, on the set of delta-QVT features; and
classify the region of tissue as a responder or non-responder based, at least in part, on the probability.

### Claim 2 (depends on 1)

The apparatus of claim 1, where the segmentation circuit is configured to annotate the tumor represented in the pre-treatment image or the at least one digitized CT post-treatment image using an ensemble segmentation approach, a level set model, or a combined spectral embedding/active contour (SEAC) approach.

### Claim 3 (depends on 1)

The apparatus of claim 1, where the segmentation circuit is configured to annotate the 3D segmented vasculature associated with the tumor represented in the pre-treatment image or the at least one digitized CT post-treatment image using a three dimensional (3D) click and grow approach.

### Claim 4 (depends on 3)

The apparatus of claim 3, where the segmentation circuit is configured to annotate the 3D segmented vasculature associated with the tumor represented in the pre-treatment image or the at least one digitized CT post-treatment image using the 3D click and grow approach by:
identifying a plurality of seed points within a volume of interest, where a member of the plurality of seed points has an intensity, where the volume of interest is in the tumor;
computing an intensity similarity between a first member of the plurality of seed points and a second, different member of the plurality of seed points; and
growing the volume of interest using a 3D region growing approach based, at least in part, on the intensity similarity.

### Claim 5 (depends on 1)

The apparatus of claim 1, where the segmentation circuit is configured to identify the center line of the 3D segmented vasculature represented in the pre-treatment image or the at least one digitized CT post-treatment image using a fast marching approach.

### Claim 6 (depends on 1)

The apparatus of claim 1, where the registration circuit registers the pre-treatment image with the at least one digitized CT post-treatment image using an affine registration approach, a rigid registration approach, or a deformable registration approach.

### Claim 7 (depends on 1)

The apparatus of claim 1, where the QVT feature extraction circuit selects the set of pre-treatment QVT features and the set of post-treatment QVT features based on a threshold level of reliability and a threshold level of reproducibility, where the QVT feature extraction circuit computes the threshold level of reliability and the threshold level of reproducibility using a concordance correlation coefficient (CCC) approach.

### Claim 8 (depends on 7)

The apparatus of claim 7, where the where the set of pre-treatment QVT features includes a maximum curvature of vessels branch feature, a standard deviation of vessel torsion feature, and a mean curvature feature, and where the set of post-treatment QVT features includes a maximum curvature of vessels branch feature, a standard deviation of vessel torsion feature, and a mean curvature feature.

### Claim 9 (depends on 1)

The apparatus of claim 1, where the classification circuit includes a machine learning classifier, where the machine learning classifier is a support vector machine (SVM) having three kernels, where the three kernels include a linear kernel, a radial basis function (RBF) kernel, and a polynomial kernel.

### Claim 10 (depends on 1)

The apparatus of claim 1, where the classification circuit includes a machine learning classifier, where the machine learning classifier is a discriminant analysis (DA) classifier, a nearest neighbor (NN) classifier, a convolutional neural network (CNN), or a random forest (RF) classifier.

### Claim 11 (depends on 1)

The apparatus of claim 1, the set of circuits further comprising an immunotherapy treatment plan circuit configured to generate an NSCLC immunotherapy treatment plan based, at least in part, on the classification and at least one of the probability, the set of delta-QVT features, the pre-treatment image, or the at least one digitized CT post-treatment image, where the NSCLC immunotherapy treatment plan defines an immunotherapy drug dosage amount and an immunotherapy drug dosage schedule.

### Claim 12 (depends on 11)

The apparatus of claim 11, the immunotherapy treatment plan circuit configured to provide the NSCLC immunotherapy treatment plan to a personalized medicine system or an immunotherapy administration system.

### Claim 13 (depends on 1)

The apparatus of claim 1, further comprising a training circuit configured to train the classification circuit, where the training circuit is configured to:
access a dataset of CT images of a region of tissue demonstrating NSCLC, where a first member of the dataset is a pre-treatment image, and a second member of the dataset is a post-treatment image of the region represented in the first member;
select a set of training delta-QVT features from the dataset, where the set of training delta-QVT features includes a QVT feature selected from the first member, and a corresponding QVT feature selected from the second member, where the set of training delta-QVT features is selected based on a level of discriminability and on a level of stability, where the level of stability is computed using a concordance correlation coefficient (CCC) approach; and
train the classification circuit using the set of delta-QVT features.

### Claim 14

A non-transitory computer-readable storage device storing computer-executable instructions that when executed by a computer control the computer to perform a method for predicting non-small cell lung cancer (NSCLC) patient response to immunotherapy, the method comprising:
accessing a pre-treatment image of a region of tissue in a patient demonstrating NSCLC, where the pre-treatment image includes a plurality of voxels;
annotating a tumor represented in the pre-treatment image;
generating a pre-treatment three dimensional (3D) segmented vasculature by segmenting a vessel associated with the tumor represented in the pre-treatment image from the tumor represented in the pre-treatment image;
identifying a center line of the pre-treatment 3D segmented vasculature;
accessing at least one post-treatment image of the region of tissue demonstrating NSCLC, where the at least one post-treatment image includes the tumor, and where the post-treatment image includes a plurality of voxels;
annotating the tumor represented in the at least one post-treatment image;
generating a post-treatment 3D segmented vasculature by segmenting a vessel associated with the tumor represented in the at least one post-treatment image from the tumor represented in the at least one post-treatment image;
identifying a center line of the post-treatment 3D segmented vasculature;
generating a registered image by registering the pre-treatment image with the at least one post-treatment image;
extracting a set of pre-treatment quantitative vessel tortuosity (QVT) features from the registered image based, at least in part on the center line of the pre-treatment 3D segmented vasculature;
extracting a set of post-treatment QVT features from the registered image based, at least in part on the center line of the post-treatment 3D segmented vasculature;
generating a set of delta-QVT features by computing, on a per-voxel basis, a difference between the set post-treatment QVT features and the set of pre-treatment QVT features;
providing the set of delta-QVT features to a machine learning classifier;
receiving, from the machine learning classifier, a probability that the region of tissue will respond to immunotherapy based, at least in part, on the set of delta-QVT features;
classifying the region of tissue as a responder or non-responder based, at least in part, on the probability; and
generating an NSCLC immunotherapy treatment plan based, at least in part, on the classification and at least one of the probability, the set of delta-QVT features, the pre-treatment image, the at least one post-treatment image, or the registered image.

### Claim 15 (depends on 14)

The non-transitory computer-readable storage device of claim 14, where the pre-treatment image is a computed tomography (CT) image of a first patient, and where the at least one post-treatment image is a CT image of the first patient.

### Claim 16 (depends on 14)

The non-transitory computer-readable storage device of claim 14, where the at least one post-treatment image is acquired at least a first time interval after administration of immunotherapy to the patient.

### Claim 17 (depends on 14)

The non-transitory computer-readable storage device of claim 14, where registering the pre-treatment image with the at least one post-treatment image includes registering the pre-treatment image with the at least one post-treatment image using an affine registration approach, a rigid registration approach, or a deformable registration approach.

### Claim 18 (depends on 14)

The non-transitory computer-readable storage device of claim 14, where the set of pre-treatment QVT features includes a maximum curvature of vessels branch feature, a standard deviation of vessel torsion feature, and a mean curvature feature, and where the set of post-treatment QVT features includes a maximum curvature of vessels branch feature, a standard deviation of vessel torsion feature, and a mean curvature feature.

### Claim 19 (depends on 14)

The non-transitory computer-readable storage device of claim 14, where the NSCLC immunotherapy treatment plan defines an immunotherapy drug dosage amount and an immunotherapy drug dosage schedule.

### Claim 20 (depends on 19)

The non-transitory computer-readable storage device of claim 19, the method further comprising:
controlling a personalized medicine system to display the NSCLC immunotherapy treatment plan, the classification, the probability, the set of delta-QVT features, the pre-treatment image, the at least one post-treatment image, or the registered image.

### Claim 21 (depends on 14)

The non-transitory computer-readable storage device of claim 14, the method further comprising training the machine learning classifier.

### Claim 22 (depends on 21)

The non-transitory computer-readable storage device of claim 21, where training the machine learning classifier includes:
accessing a set of CT images of a population of NSCLC patients, where the population includes a set of responders and a set of non-responders, where a member of the set of CT images includes a voxel, where the set of CT images includes at least one pre-treatment image associated with a responder member of the population and at least one post-treatment image associated with the responder member, and at least one pre-treatment image associated with a non-responder member of the population and at least one post-treatment image associated with the non-responder member;
dividing the set of CT images into a training set and a testing set, where the training set includes at least one pre-treatment CT image and at least one post-treatment CT image associated with a responder, and at least one pre-treatment CT image and at least one post-treatment CT image associated with a non-responder, and where the testing set includes at least one pre-treatment CT image and at least one post-treatment CT image associated with a responder, and at least one pre-treatment CT image and at least one post-treatment CT image associated with a non-responder;
extracting a set of training pre-treatment QVT features and a set of training post-treatment QVT features from the training set;
generating a set of training delta-QVT features by computing, on a per-voxel basis, a difference between the set of training post-treatment QVT features and the set of training pre-treatment QVT features;
identifying a set of stable delta-QVT features in the set of training delta-QVT features, where the set of stable delta-QVT features are selected based on a threshold level of reliability and a threshold level of reproducibility using a concordance correlation coefficient (CCC) approach; and
training the machine learning classifier using the set of stable delta-QVT features.

### Claim 23 (depends on 22)

The non-transitory computer-readable storage device of claim 22, the method further comprising testing the machine learning classifier using the testing set.

### Claim 24 (depends on 14)

The non-transitory computer-readable storage device of claim 14, where the machine learning classifier is a support vector machine (SVM).

### Claim 25 (depends on 24)

The non-transitory computer-readable storage device of claim 24, where the SVM has three kernels, where the three kernels include a linear kernel, a radial basis function (RBF) kernel, and a polynomial kernel.

### Claim 26 (depends on 14)

The non-transitory computer-readable storage device of claim 14, where the machine learning classifier is a discriminant analysis (DA) classifier, a nearest neighbor (NN) classifier, a convolutional neural network (CNN), or a random forest (RF) classifier.

### Claim 27 (depends on 14)

The non-transitory computer-readable storage device of claim 14, where the set of pre-treatment QVT features and the set of post-treatment QVT features are selected using a minimum redundancy maximum relevance (mRMR) approach, a T-test approach, or a Wilcoxon rank sum test.

### Claim 28 (depends on 14)

The non-transitory computer-readable storage device of claim 14, where the set of delta-QVT features are selected based on a threshold level of reliability and a threshold level of reproducibility.

### Claim 29 (depends on 28)

The non-transitory computer-readable storage device of claim 28, where the threshold level of reliability and the threshold level of reproducibility are computed using a concordance correlation coefficient (CCC) approach.

### Claim 30

A computer-readable storage device storing computer-executable instructions that, in response to execution, cause an immunotherapy prediction response system to perform operations comprising:
accessing a pre-treatment radiological image of a region of tissue demonstrating non-small cell lung cancer (NSCLC), where the pre-treatment radiological image includes a tumor and a vasculature associated with the tumor, where the pre-treatment radiological image includes a plurality of voxels;
accessing at least one post-treatment radiological image of the region of tissue demonstrating NSCLC, where the at least one post-treatment radiological image includes the tumor and the vasculature, where the post-treatment radiological image includes a plurality of voxels;
generating a registered image by registering the pre-treatment radiological image with the at least one post-treatment radiological image;
extracting a set of pre-treatment quantitative vessel tortuosity (QVT) features from the vasculature represented in the registered image;
extracting a set of post-treatment QVT features from the vasculature represented in the registered image;
generating a set of delta-QVT features by computing a difference between the set of post-treatment QVT features and the set of pre-treatment QVT features, where a member of the set of delta-QVT features has a threshold level of stability;
providing the set of delta-QVT features to a machine learning classifier;
receiving, from the machine learning classifier, a probability that the region of tissue will respond to immunotherapy based, at least in part, on the set of delta-QVT features;
classifying the region of tissue as a responder or non-responder based, at least in part, on the probability; and
generating an NSCLC immunotherapy treatment plan based, at least in part, on the classification and at least one of the probability, the set of delta-QVT features, the pre-treatment radiological image, or the at least one post-treatment radiological image.

### Claim 31

A computer-readable storage device storing computer-executable instructions that, in response to execution, cause an immunotherapy prediction response system to perform operations comprising:
accessing a registered radiological image of a region of tissue demonstrating non-small cell lung cancer (NSCLC), where the registered radiological image includes a pre-treatment radiological image of the region of tissue registered with a post-immunotherapy radiological image of the region of tissue, where the registered radiological image includes a tumor and a vasculature associated with the tumor, where the registered radiological image includes a plurality of voxels;
extracting a set of pre-treatment quantitative vessel tortuosity (QVT) features from the vasculature represented in the registered radiological image;
extracting a set of post-treatment QVT features from the vasculature represented in the registered radiological image;
generating a set of delta-QVT features by computing a difference between the set of post-treatment QVT features and the set of pre-treatment QVT features, where a member of the set of delta-QVT features has a threshold level of stability;
computing a probability that the region of tissue will respond to immunotherapy based, at least in part, on the set of delta-QVT features;
classifying the region of tissue as a responder or non-responder based, at least in part, on the probability; and
generating an NSCLC immunotherapy treatment plan based, at least in part, on the classification and at least one of the probability, the set of delta-QVT features, or the registered radiological image.