# US 10,769,783 B2 · Predicting prostate cancer biochemical recurrence using combined nuclear NF-KB/P65 localization and gland morphology > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,769,783 B2 - Document ID: US10769783B2 - Status: Granted - Kind: B2 - Application number: 16/217,274 - Filed: 2018-12-12 - Published: 2019-08-15 - Granted: 2020-09-08 - Earliest claimed: 2018-02-09 - Assignees: Case Western Reserve University - Applicants: Case Western Reserve University - Inventors: Anant Madabhushi; Patrick Leo; Andrew Janowczyk; Sanjay Gupta - Web page: https://patentgrep.com/patent/US10769783B2 ## CPC - G06T 7/0012: {Biomedical image inspection} - G06F 18/23213: with fixed number of clusters, e.g. K-means clustering - G06F 18/2415: based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate - G06V 20/695: {Preprocessing, e.g. image segmentation} - G06T 2207/20036: Morphological image processing - G06T 2207/20081: Training; Learning - G06T 2207/30024: Cell structures in vitro; Tissue sections in vitro - G06T 2207/30081: Prostate - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Embodiments include controlling a processor to perform operations for predicting biochemical recurrence (BCR) in prostate cancer (PCa), including accessing a first digitized pathology slide having a first stain channel of a region of tissue demonstrating PCa; accessing a second digitized pathology slide having a second, different stain channel of the region of tissue; extracting morphology features from the first stain channel; extracting stain intensity features from the second stain channel, where a stain intensity feature quantifies an amount of a molecular biomarker present in a cellular nucleus; controlling a first machine learning classifier to generate a first probability of BCR based on the morphology features; controlling a second machine learning classifier to generate a second, different probability of BCR based on the stain intensity features; computing an aggregate probability of BCR based on the first probability and the second probability; and displaying the aggregate probability. ## Claims ### Claim 1 A non-transitory computer-readable storage device storing instructions that when executed control a processor to perform operations for predicting biochemical recurrence (BCR) in a prostate cancer (PCa) patient, the operations comprising: accessing a first digitized pathology slide of a region of tissue demonstrating PCa, the first digitized pathology slide having a first stain channel; accessing a second, different digitized pathology slide of the region of tissue, the second digitized pathology slide having a second, different stain channel; extracting a set of morphology features from the first stain channel of the first digitized pathology slide; extracting a set of stain intensity features from the second, different stain channel of the second, different digitized pathology slide, where a stain intensity feature quantifies an amount of a molecular biomarker present in a cellular nucleus represented in the second, different digitized pathology slide; controlling a first machine learning classifier to generate a first probability of BCR based on the set of morphology features; controlling a second, different machine learning classifier to generate a second, different probability of BCR based on the set of stain intensity features; computing an aggregate probability of BCR as a function of the first probability and the second, different probability; and displaying the aggregate probability. ### Claim 2 (depends on 1) The non-transitory computer-readable storage device of claim 1, where the first digitized pathology slide is a digitized image of a first histology slice of a region of tissue demonstrating PCa, where the first histology slice is a hematoxylin and eosin (H&E) stained histology slice, where the first digitized pathology slide has a plurality of pixels, a pixel having an intensity. ### Claim 3 (depends on 2) The non-transitory computer-readable storage device of claim 2, where extracting the set of morphology features from the first stain channel of the first digitized pathology slide includes: selecting a plurality of pixels in the first digitized pathology slide; clustering members of the plurality of pixels in red-green-blue (RGB) space using k-means clustering, where k=4; labeling a cluster as stroma, cytoplasm, nuclei, or lumen; classifying a pixel according to the cluster the pixel is nearest; and segmenting a boundary of a lumen cluster. ### Claim 4 (depends on 3) The non-transitory computer-readable storage device of claim 3, where the set of morphology features includes a gland orientation disorder average range feature, a gland orientation disorder information measure 2 feature, and a gland orientation disorder energy range feature. ### Claim 5 (depends on 3) The non-transitory computer-readable storage device of claim 3, where the set of morphology features includes at least one gland lumen feature and at least one Haralick feature. ### Claim 6 (depends on 5) The non-transitory computer-readable storage device of claim 5, where the at least one gland lumen features includes a global graph feature, a shape feature, a disorder feature, or a sub-graph feature, and where the at least one Haralick feature includes a pixel intensity contrast feature, an energy feature, or an entropy feature. ### Claim 7 (depends on 1) The non-transitory computer-readable storage device of claim 1, where the molecular biomarker is NF-KB/p65. ### Claim 8 (depends on 7) The non-transitory computer-readable storage device of claim 7, where the second, different digitized pathology slide is a digitized image of a second histology slice stained using 3,3-Diaminobenzidine (DAB), where the second, different digitized pathology slide has a plurality of pixels, a pixel having an intensity. ### Claim 9 (depends on 8) The non-transitory computer-readable storage device of claim 8, where extracting the set of stain intensity features includes: segmenting a cellular nucleus represented in the second different digitized pathology slide; and computing an optical density of the DAB stain in a pixel of a segmented cellular nucleus. ### Claim 10 (depends on 9) The non-transitory computer-readable storage device of claim 9, where the set of stain intensity features includes: a percentage of cellular nuclei pixels that have a negative DAB stain optical intensity, a ratio of a number of cellular nuclei pixels having a moderately positive DAB stain optical intensity to a number of cellular nuclei pixels having a strongly positive DAB stain optical intensity, and a ratio of a number of cellular nuclei pixels having a weakly positive DAB stain optical intensity to the number of cellular nuclei pixels having a strongly positive DAB stain optical intensity. ### Claim 11 (depends on 9) The non-transitory computer-readable storage device of claim 9, where segmenting the cellular nucleus represented in the second different digitized pathology slide includes segmenting the cellular nucleus using a deep learning segmentation approach that employs a 65 pixel by 65 pixel patch. ### Claim 12 (depends on 1) The non-transitory computer-readable storage device of claim 1, where the first probability of BCR ranges from 0 to 1, where the first probability represents a likelihood that the patient will experience early BCR, and where the second probability of BCR ranges from 0 to 1, where the second probability represents the likelihood that the patient will experience early BCR. ### Claim 13 (depends on 1) The non-transitory computer-readable storage device of claim 1, where the aggregate probability predicts BCR with an accuracy of at least 0.87. ### Claim 14 (depends on 1) The non-transitory computer-readable storage device of claim 1, the operations further comprising generating a personalized cancer treatment plan based, at least in part, on the aggregate probability. ### Claim 15 An apparatus for predicting biochemical recurrence (BCR) in prostate cancer (PCa) patients, the apparatus comprising: a processor; a memory configured to store at least one digitized pathology slide of a region of tissue demonstrating PCa; an input/output (I/O) interface; a set of circuits; and an interface that connects the processor, the memory, the I/O interface, and the set of circuits, the set of circuits comprising: an image acquisition circuit configured to: access a first digitized pathology slide of a region of tissue demonstrating PCa, the first digitized pathology slide having a first stain channel, and access a second, different digitized pathology slide of the region of tissue, the second digitized pathology slide having a second, different stain channel; a morphology feature extraction circuit configured to: extract a set of morphology features from the first stain channel of the first digitized pathology slide, where the set of morphology features includes at least one gland lumen feature and at least one Haralick feature, and compute a first probability that the region of tissue will experience BCR based, at least in part, on the set of morphology features; a molecular biomarker feature extraction circuit configured to: extract a set of stain intensity features from the second stain channel of the second digitized pathology slide, where a stain intensity feature quantifies an amount of a molecular biomarker present in a cellular nucleus represented in the second, different digitized pathology slide, where the molecular biomarker is NF-KB/p65, and compute a second probability that the region of tissue will experience BCR based, at least in part, on the set of stain intensity features; a BCR recurrence classification circuit configured to compute an aggregate probability that the region of tissue will experience BCR as a function of the first probability and the second probability; and a display circuit configured to display the aggregate probability and at least one of the first digitized pathology slide, the second digitized pathology slide, the first probability, the second probability, the set of morphology features, or the set of stain intensity features. ### Claim 16 (depends on 15) The apparatus of claim 15, where: the first digitized pathology slide is a digitized image of a first histology slice of a region of tissue demonstrating PCa, where the first histology slice is a hematoxylin and eosin (H&E) stained histology slice, where the first digitized pathology slide has a plurality of pixels, a pixel having an intensity; and where the second, different digitized pathology slide is a digitized image of a second histology slice stained using 3,3-Diaminobenzidine (DAB), where the second, different digitized pathology slide has a plurality of pixels, a pixel having an intensity. ### Claim 17 (depends on 16) The apparatus of claim 16, where the morphology circuit is configured to extract the set of morphology features by: selecting a plurality of pixels in the first digitized pathology slide; clustering members of the plurality of pixels in red-green-blue (RGB) space using k-means clustering, where k=4; labeling a cluster as stroma, cytoplasm, nuclei, or lumen; classifying a pixel according to the cluster the pixel is nearest; and segmenting a boundary of a lumen cluster. ### Claim 18 (depends on 16) The apparatus of claim 16, where the set of stain intensity features includes: a percentage of cellular nuclei pixels that have a negative DAB stain optical intensity, a ratio of the number of cellular nuclei pixels having a moderately positive DAB stain optical intensity to the number of cellular nuclei pixels having a strongly positive DAB stain optical intensity, and a ratio of the number of cellular nuclei pixels having a weakly positive DAB stain optical intensity to the number of cellular nuclei pixels having a strongly positive DAB stain optical intensity. ### Claim 19 (depends on 15) The apparatus of claim 15, further comprising a personalized treatment plan circuit configured to generate a personalized treatment plan based, at least in part, on the aggregate probability. ### Claim 20 A method for predicting biochemical recurrence (BCR) in prostate cancer (PCa) patients, the method comprising: accessing a first digitized pathology slide of a region of tissue demonstrating PCa, the first digitized pathology slide having a first stain channel; accessing a second, different digitized pathology slide of the region of tissue, the second digitized pathology slide having a second, different stain channel; extracting a set of morphology features from the first stain channel of the first digitized pathology slide; extracting a set of optical density features from the second, different stain channel of the second, different digitized pathology slide, where an optical density feature quantifies an amount of a BCR-predictive molecular biomarker present in a cellular nucleus represented in the second, different digitized pathology slide; controlling a first machine learning classifier to generate a first probability of BCR based on the set of morphology features; controlling a second, different machine learning classifier to generate a second, different probability of BCR based on the set of optical density features; and computing an aggregate probability of BCR as a function of the first probability and the second, different probability.