# US 2021/0279864 A1 · Distinguishing colon cancer stages based on computationally derived morphological features of cancer nuclei

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0279864 A1
- Document ID: US20210279864A1
- Status: Published application
- Kind: A1
- Application number: 17/140,586
- Filed: 2021-01-04
- Published: 2021-09-09
- Earliest claimed: 2020-03-04
- Assignees: Case Western Reserve University; Louis Stokes Cleveland Veterans Administration Medical Center
- Applicants: Case Western Reserve University; Louis Stokes Cleveland Veterans Administration Medical Center
- Inventors: Anant Madabhushi; Neeraj Kumar; Joseph E. Willis
- Web page: https://patentgrep.com/patent/US20210279864A1

## CPC

- G06T 7/0012: {Biomedical image inspection}
- G06T 7/11: Region-based segmentation
- G06T 7/41: based on statistical description of texture
- G06T 7/62: of area, perimeter, diameter or volume
- G16H 10/40: for data related to laboratory analysis, e.g. patient specimen analysis
- G16H 30/40: for processing medical images, e.g. editing
- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- G16H 50/70: for mining of medical data, e.g. analysing previous cases of other patients
- G16H 70/60: relating to pathologies
- G06T 2207/10056: Microscopic image
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30024: Cell structures in vitro; Tissue sections in vitro
- G06T 2207/30028: Colon; Small intestine
- G06T 2207/30081: Prostate
- G06T 2207/30096: Tumor; Lesion
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

Embodiments discussed herein facilitate determination of cancer stages based at least in part on shape, size, and/or texture features of cancer nuclei. One example embodiment is a method, comprising: accessing at least a portion of a digital whole slide image (WSI) comprising a tumor; segmenting (e.g., via a first deep learning model) the tumor on the at least the portion of the digital WSI; segmenting (e.g., via a second deep learning model) cancer nuclei in the segmented tumor; extracting one or more features from the segmented cancer nuclei; providing the one or more features extracted from the segmented cancer nuclei to a trained machine learning model; and receiving, from the machine learning model, an indication of a cancer stage of the tumor.

## Claims

### Claim 1

A non-transitory computer-readable medium storing computer-executable instructions that, when executed, cause a processor to perform operations, comprising:
accessing at least a portion of a digital whole slide image (WSI) comprising a tumor;
segmenting the tumor on the at least the portion of the digital WSI;
segmenting cancer nuclei in the segmented tumor;
extracting one or more features from the segmented cancer nuclei;
providing the one or more features extracted from the segmented cancer nuclei to a trained machine learning model; and
receiving, from the machine learning model, an indication of a cancer stage of the tumor.

### Claim 2 (depends on 1)

The non-transitory computer-readable medium of claim 1, wherein the machine learning model is one of, or an ensemble of two or more of, a logistic regression model, a Cox regression model, a Least Absolute Shrinkage and Selection Operator (LASSO) regression model, a naïve Bayes classifier, a support vector machine (SVM) with a linear kernel, a SVM with a radial basis function (RBF) kernel, a linear discriminant analysis (LDA) classifier, a quadratic discriminant analysis (QDA) classifier, a logistic regression classifier, a decision tree, a random forest, a diagonal LDA, a diagonal QDA, a neural network, an AdaBoost algorithm, an elastic net, a Gaussian process classification, or a nearest neighbors classification.

### Claim 3 (depends on 1)

The non-transitory computer-readable medium of claim 1, wherein the one or more features comprise at least one of: a statistic of at least one shape feature of the segmented cancer nuclei, the statistic of at least one size feature of the segmented cancer nuclei, or the statistic of at least one texture feature of the segmented cancer nuclei.

### Claim 4 (depends on 3)

The non-transitory computer-readable medium of claim 3, wherein the statistic is one of a mean, a median, a standard deviation, a skewness, a kurtosis, a range, a minimum, a maximum, a percentile, or histogram frequencies.

### Claim 5 (depends on 3)

The non-transitory computer-readable medium of claim 3, wherein the at least one size feature of the segmented cancer nuclei comprises one or more of an area, a major axis length, or a perimeter.

### Claim 6 (depends on 1)

The non-transitory computer-readable medium of claim 1, wherein segmenting the tumor on the at least the portion of the digital WSI comprises segmenting the tumor on the at least the portion of the digital WSI via a first deep learning (DL) model.

### Claim 7 (depends on 6)

The non-transitory computer-readable medium of claim 6, wherein the first DL model is a fully convolutional deep neural network with a Visual Geometry Group (VGG)-18 architecture.

### Claim 8 (depends on 6)

The non-transitory computer-readable medium of claim 6, wherein the first DL model employs one of the following optimizers: an Adam optimizer, a stochastic gradient descent (SGD) optimizer, a SGD optimizer with momentum, or a SGD optimizer with Nesterov momentum.

### Claim 9 (depends on 1)

The non-transitory computer-readable medium of claim 1, wherein segmenting cancer nuclei in the segmented tumor comprises segmenting cancer nuclei in the segmented tumor via a second deep learning (DL) model.

### Claim 10 (depends on 9)

The non-transitory computer-readable medium of claim 9, wherein the first DL model is a Mask-RCNN (Regional Convolutional Neural Network) with a Residual Network (Resnet)-50 architecture.

### Claim 11 (depends on 9)

The non-transitory computer-readable medium of claim 9, wherein the second DL model employs one of the following optimizers: an Adam optimizer, a stochastic gradient descent (SGD) optimizer, a SGD optimizer with momentum, or a SGD optimizer with Nesterov momentum.

### Claim 12 (depends on 1)

The non-transitory computer-readable medium of claim 1, wherein the cancer is prostate cancer.

### Claim 13 (depends on 12)

The non-transitory computer-readable medium of claim 12, wherein the cancer stage is one of stage 2 or stage 4.

### Claim 14

A non-transitory computer-readable medium storing computer-executable instructions that, when executed, cause a processor to perform operations, comprising:
accessing a training set comprising a plurality of digital whole slide images (WSIs), wherein each digital WSI of the plurality of digital WSIs comprises an associated tumor and a known associated cancer stage;
for each digital WSI of the training set:
segmenting the associated tumor on that digital WSI;
segmenting associated cancer nuclei in the associated segmented tumor of that digital WSI; and
extracting an associated value for each of a plurality of features from the associated segmented cancer nuclei of that digital WSI;
determining a set of best features from the plurality of features, based at least in part on the known associated cancer stage for each digital WSI of the training set and on the associated values for each of the plurality of features for each digital WSI of the training set; and
constructing a machine learning model configured to determine an additional cancer stage for an additional digital WSI based at least in part on the set of best features.

### Claim 15 (depends on 14)

The non-transitory computer-readable medium of claim 14, wherein determining the set of best features comprises determining the set of best features via a Wilcoxon rank-sum test.

### Claim 16 (depends on 14)

The non-transitory computer-readable medium of claim 14, wherein the machine learning model is one of, or an ensemble of two or more of, a logistic regression model, a Cox regression model, a Least Absolute Shrinkage and Selection Operator (LASSO) regression model, a naïve Bayes classifier, a support vector machine (SVM) with a linear kernel, a SVM with a radial basis function (RBF) kernel, a linear discriminant analysis (LDA) classifier, a quadratic discriminant analysis (QDA) classifier, a logistic regression classifier, a decision tree, a random forest, a diagonal LDA, a diagonal QDA, a neural network, an AdaBoost algorithm, an elastic net, a Gaussian process classification, or a nearest neighbors classification.

### Claim 17 (depends on 14)

The non-transitory computer-readable medium of claim 14, wherein the one or more features comprise at least one of: a statistic of at least one shape feature of the segmented cancer nuclei, the statistic of at least one size feature of the segmented cancer nuclei, or the statistic of at least one texture feature of the segmented cancer nuclei.

### Claim 18 (depends on 17)

The non-transitory computer-readable medium of claim 17, wherein the statistic is one of a mean, a median, a standard deviation, a skewness, a kurtosis, a range, a minimum, a maximum, a percentile, or histogram frequencies.

### Claim 19 (depends on 17)

The non-transitory computer-readable medium of claim 17, wherein the at least one size feature of the segmented cancer nuclei comprises one or more of an area, a major axis length, or a perimeter.

### Claim 20 (depends on 14)

The non-transitory computer-readable medium of claim 14, wherein, for each digital WSI of the training set, segmenting the associated tumor on that digital WSI comprises segmenting the associated tumor on that digital WSI via a first deep learning (DL) model.

### Claim 21 (depends on 14)

The non-transitory computer-readable medium of claim 14, wherein, for each digital WSI of the training set, segmenting the associated cancer nuclei in the associated segmented tumor of that digital WSI comprises segmenting the associated cancer nuclei in the associated segmented tumor of that digital WSI via a second deep learning (DL) model.

### Claim 22

An apparatus, comprising:
memory configured to store at least a portion of a digital whole slide image (WSI) comprising a tumor;
one or more processors configured to perform operations comprising:
segmenting the tumor on the at least the portion of the digital WSI;
segmenting cancer nuclei in the segmented tumor;
extracting one or more features from the segmented cancer nuclei;
providing the one or more features extracted from the segmented cancer nuclei to a trained machine learning model; and
receiving, from the machine learning model, an indication of a cancer stage of the tumor.

### Claim 23 (depends on 22)

The apparatus of claim 22, wherein the machine learning model is one of, or an ensemble of two or more of, a logistic regression model, a Cox regression model, a Least Absolute Shrinkage and Selection Operator (LASSO) regression model, a naïve Bayes classifier, a support vector machine (SVM) with a linear kernel, a SVM with a radial basis function (RBF) kernel, a linear discriminant analysis (LDA) classifier, a quadratic discriminant analysis (QDA) classifier, a logistic regression classifier, a decision tree, a random forest, a diagonal LDA, a diagonal QDA, a neural network, an AdaBoost algorithm, an elastic net, a Gaussian process classification, or a nearest neighbors classification.

### Claim 24 (depends on 22)

The apparatus of claim 22, wherein the one or more features comprise at least one of: a statistic of at least one shape feature of the segmented cancer nuclei, the statistic of at least one size feature of the segmented cancer nuclei, or the statistic of at least one texture feature of the segmented cancer nuclei.

### Claim 25 (depends on 24)

The apparatus of claim 24, wherein the statistic is one of a mean, a median, a standard deviation, a skewness, a kurtosis, a range, a minimum, a maximum, a percentile, or histogram frequencies.

### Claim 26 (depends on 24)

The apparatus of claim 24, wherein the at least one size feature of the segmented cancer nuclei comprises one or more of an area, a major axis length, or a perimeter.

### Claim 27 (depends on 22)

The apparatus of claim 22, wherein segmenting the tumor on the at least the portion of the digital WSI comprises segmenting the tumor on the at least the portion of the digital WSI via a first deep learning (DL) model.

### Claim 28 (depends on 22)

The apparatus of claim 22, wherein segmenting cancer nuclei in the segmented tumor comprises segmenting cancer nuclei in the segmented tumor via a second deep learning (DL) model.