# US 2021/0241121 A1 · Systems Configured for Cell-Based Histopathological Learning and Prediction and Methods Thereof > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0241121 A1 - Document ID: US20210241121A1 - Status: Published application - Kind: A1 - Application number: 17/168,791 - Filed: 2021-02-05 - Published: 2021-08-05 - Earliest claimed: 2020-02-05 - Assignees: Origin Labs, Inc. - Applicants: Origin Labs, Inc. - Inventors: Darick M. Tong; Nishant Borude; Nivedita Suresh; Evan Szu; Clifford Szu - Web page: https://patentgrep.com/patent/US20210241121A1 ## CPC - G06N 3/088: Non-supervised learning, e.g. competitive learning - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/047: Probabilistic or stochastic networks - G06N 3/0475: Generative networks - G06N 3/09: Supervised learning - G06N 3/094: Adversarial learning - G06T 7/0014: {using an image reference approach} - G16H 10/40: for data related to laboratory analysis, e.g. patient specimen analysis - G16H 30/40: for processing medical images, e.g. editing - G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems - G06N 3/048: Activation functions - G06T 2207/10056: Microscopic image - G06T 2207/10061: from scanning electron microscope - G06T 2207/20081: Training; Learning - G06T 2207/20084: Artificial neural networks [ANN] - G06T 2207/30024: Cell structures in vitro; Tissue sections in vitro - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA ## Abstract Histopathological scoring can be based on ratios of different types of cells, and in particular, cells which exhibit a particular genotypic or phenotypic characteristic, as identified by a biological assay. Automating the scoring process with an image analysis algorithm requires both correctly delineating cells, a process known as segmentation, and classifying each cell according to its morphology and reactivity to the assay. Successful classification thus depends on both successful segmentation and successful classification, resulting in the error rates of the two steps being compounded. Systems and methods of the present disclosure reduce error by performing the cell counting and classification task in a single step using a generative adversarial network (or GAN). The present disclosure similarly employs a GAN for counting cells by representing the training data as a Gaussian at the center of each cell nucleus. ## Claims ### Claim 1 A method comprising: receiving, by at least one processor, a tissue image comprising a digital representation of a plurality of cells of a tissue; utilizing, by the at least one processor, a histopathological score prediction model to predict at least one mask delineating cells in the tissue image according to cell staining based on learned histopathological scoring parameters; wherein each mask of the at least one mask is associated with each image band of at least one image band; wherein each mask of the at least one mask comprises pixel values representative of each image band of the at least one image band; wherein each image band of the at least one image band represents a cell type classification of at least one cell type classification; wherein the pixel values of each mask comprises a Gaussian distribution of pixel values centered at each cell of each cell type classification of the at least one cell type classification; determining, by the at least one processor, a sum of the Gaussian distribution of the pixel values of each mask of the at least one mask; wherein the sum of the Gaussian distribution of the pixel values of each mask represents a count of cells of each cell type classification; determining, by the at least one processor, a histopathological score based at least in part on the count of cells of each cell type classification; and causing to display, by at least one processor, the histopathological score on at least one screen of at least one computing device associated with at least one user. ### Claim 2 (depends on 1) The method as recited in claim 1, wherein the histopathological score prediction model comprises a generative adversarial network (GAN). ### Claim 3 (depends on 1) The method as recited in claim 1, wherein the at least one image band comprises a plurality of grayscale bands. ### Claim 4 (depends on 1) The method as recited in claim 1, further comprising determining, by the at least one processor, a cell-type-specific histopathological score for a particular cell type classification of the at least one cell types classification based at least in part on a ratio of the sum of a mask associated with the particular cell type classification to a total sum of at least one mask. ### Claim 5 (depends on 1) The method as recited in claim 1, further comprising: receiving, by the at least one processor, an expert annotated tissue sample image comprising a plurality of cell type classification annotations marking a center of each cell of a plurality of cells of a particular cell type classification; converting, by the at least one processor, the plurality of cell type classification annotations to a training Gaussian mask representing a plurality of true cell type classifications by applying a bivariate normal function having a parameter centered as each cell of the plurality of cells according to the plurality of cell type classification annotations; and training, by at least one processor, the histopathological score prediction model on the training Gaussian mask. ### Claim 6 (depends on 1) The method as recited in claim 1, wherein a sum of the Gaussian distribution of pixel values centered at each cell of each cell type classification is equal to 1. ### Claim 7 (depends on 1) The method as recited in claim 1, wherein the histopathological score prediction model comprises a Gaussian function to define the Gaussian distribution of the pixel values centered at each cell of each cell type classification of the at least one cell type classification; wherein Gaussian function comprises parameters for expected value and variance that are customized for a size of cells of each cell type classification. ### Claim 8 A system comprising: at least one processor in communication with at least one memory and configured to access instructions stored in the memory that cause the at least one processor to perform steps to: receive a tissue image comprising a digital representation of a plurality of cells of a tissue; utilize a histopathological score prediction model to predict at least one mask delineating cells in the tissue image according to cell staining based on learned histopathological scoring parameters; wherein each mask of the at least one mask is associated with each image band of at least one image band; wherein each mask of the at least one mask comprises pixel values representative of each image band of the at least one image band; wherein each image band of the at least one image band represents a cell type classification of at least one cell type classification; wherein the pixel values of each mask comprises a Gaussian distribution of pixel values centered at each cell of each cell type classification of the at least one cell type classification; determine a sum of the Gaussian distribution of the pixel values of each mask of the at least one mask; wherein the sum of the Gaussian distribution of the pixel values of each mask represents a count of cells of each cell type classification; determine a histopathological score based at least in part on the count of cells of each cell type classification; and cause to display the histopathological score on at least one screen of at least one computing device associated with at least one user. ### Claim 9 (depends on 8) The system as recited in claim 8, wherein the histopathological score prediction model comprises a generative adversarial network (GAN). ### Claim 10 (depends on 8) The system as recited in claim 8, wherein the at least one image band comprises a plurality of grayscale bands. ### Claim 11 (depends on 8) The system as recited in claim 8, wherein the instructions further cause the at least one processor to perform steps to determine a cell-type-specific histopathological scoring for a particular cell type classification of the at least one cell type classification based at least in part on a ratio of the sum of a mask associated with the particular cell type to a total sum of at least one mask. ### Claim 12 (depends on 8) The system as recited in claim 8, wherein the instructions further cause the at least one processor to perform steps to: receive an expert annotated tissue sample image comprising a plurality of cell type classification annotations marking a center of each cell of a plurality of cells of a particular cell type classification; convert the plurality of cell type classification annotations to a training Gaussian mask representing a plurality of true cell type classifications by applying a bivariate normal function having a parameter centered as each cell of the plurality of cells according to the plurality of cell type classification annotations; and train the histopathological score prediction model on the training Gaussian mask. ### Claim 13 (depends on 8) The system as recited in claim 8, wherein a sum of the Gaussian distribution of pixel values centered at each cell of each cell type classification is equal to 1. ### Claim 14 (depends on 8) The system as recited in claim 8, wherein the histopathological score prediction model comprises a Gaussian function to define the Gaussian distribution of the pixel values centered at each cell of each cell type classification of the at least one cell type classification; wherein Gaussian function comprises parameters for expected value and variance that are customized for a size of cells of each cell type classification. ### Claim 15 A non-transitory computer readable medium having software instructions stored thereon, the software instructions configured to cause at least one processor to perform steps comprising: receiving a tissue image comprising a digital representation of a plurality of cells of a tissue; utilizing a histopathological score prediction model to predict at least one mask delineating cells in the tissue image according to cell staining based on learned histopathological scoring parameters; wherein each mask of the at least one mask is associated with each image band of at least one image band; wherein each mask of the at least one mask comprises pixel values representative of each image band of the at least one image band; wherein each image band of the at least one image band represents a cell type classification of at least one cell type classification; wherein the pixel values of each mask comprises a Gaussian distribution of pixel values centered at each cell of each cell type classification of the at least one cell type classification; determine a sum of the Gaussian distribution of the pixel values of each mask of the at least one mask; wherein the sum of the Gaussian distribution of the pixel values of each mask represents a count of cells of each cell type classification; determining a histopathological score based at least in part on the count of cells of each cell type classification; and causing to display the histopathological score on at least one screen of at least one computing device associated with at least one user. ### Claim 16 (depends on 15) The non-transitory computer readable medium as recited in claim 15, wherein the histopathological score prediction model comprises a generative adversarial network (GAN). ### Claim 17 (depends on 15) The non-transitory computer readable medium as recited in claim 15, wherein the at least one image band comprises a plurality of grayscale bands. ### Claim 18 (depends on 15) The non-transitory computer readable medium as recited in claim 15, wherein the software instructions are further configured to cause the at least one processor to perform steps comprising determining, by the at least one processor, a cell-type-specific histopathological score for a particular cell type classification of the at least one cell type classification based at least in part on a ratio of the sum of a mask associated with the particular cell type to a total sum of at least one mask. ### Claim 19 (depends on 15) The non-transitory computer readable medium as recited in claim 15, wherein the software instructions are further configured to cause the at least one processor to perform steps comprising: receiving, by the at least one processor, an expert annotated tissue sample image comprising a plurality of cell type classification annotations marking a center of each cell of a plurality of cells of a particular cell type classification; converting, by the at least one processor, the plurality of cell type classification annotations to a training Gaussian mask representing a plurality of true cell type classifications by applying a bivariate normal function having a parameter centered as each cell of the plurality of cells according to the plurality of cell type classification annotations; and training, by at least one processor, the histopathological score prediction model on the training Gaussian mask. ### Claim 20 (depends on 15) The non-transitory computer readable medium as recited in claim 15, wherein a sum of the Gaussian distribution of pixel values centered at each cell of each cell type classification is equal to 1.