# US 2021/0241122 A1 · Systems Configured for Area-Based Histopathological Learning and Prediction and Methods Thereof > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0241122 A1 - Document ID: US20210241122A1 - Status: Published application - Kind: A1 - Application number: 17/168,847 - Filed: 2021-02-05 - Published: 2021-08-05 - Earliest claimed: 2020-02-05 - Assignees: Origin Labs, Inc. - Applicants: Origin Labs, Inc. - Inventors: Darick M. Tong; Nishant Borude; Nivedita Suresh; Evan Szu; Clifford Szu - Web page: https://patentgrep.com/patent/US20210241122A1 ## CPC - G16B 40/00: ICT specially adapted for biostatistics; ICT specially adapted for bioinformatics-related machine learning or data mining, e.g. knowledge discovery or pattern finding - G16H 10/40: for data related to laboratory analysis, e.g. patient specimen analysis - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/0475: Generative networks - G06N 3/08: Learning methods - G06N 3/09: Supervised learning - G06N 3/094: Adversarial learning - G06N 20/00: Machine learning - G16B 45/00: ICT specially adapted for bioinformatics-related data visualisation, e.g. displaying of maps or networks - G16H 30/40: for processing medical images, e.g. editing - G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems - G16H 50/70: for mining of medical data, e.g. analysing previous cases of other patients - G06N 3/084: Backpropagation, e.g. using gradient descent - G06N 3/088: Non-supervised learning, e.g. competitive learning - G16B: BIOINFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR GENETIC OR PROTEIN-RELATED DATA PROCESSING IN COMPUTATIONAL MOLECULAR BIOLOGY - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract Histopathological scoring can be based on the areas of certain types of cells or the expression of genotypic or phenotypic characteristics of those cells, as identified by a biological assay. Automating a scoring process with an image analysis algorithm includes correctly delineating the areas of interest, a process known as segmentation. The present systems and methods accomplish this segmentation using a generative adversarial network trained to generate masks covering each area of interest. The invention can perform both segmentation and classification by using a separate image band for each class. A scoring algorithm may utilize the classifications of, for example, a tumor area and an area of immune cell staining by interpreting the separate image bands of each area. Classification problems with more bands would use images with the equivalent number of bands. There is no limit to the number of bands an image can encode for each pixel. ## Claims ### Claim 1 A method comprising: receiving, by at least one processor, a tissue image comprising a digital representation of a plurality of cells of a tissue; utilizing, by the at least one processor, a histopathological scoring model to predict at least one mask delineating areas of interest in the tissue image according to cell types or biomarkers based on learned histopathological scoring parameters; wherein each mask of the at least one mask comprises: i) at least one polygon formed by labelled pixels representing a delineation of each area of interest of the areas of interest, and ii) at least one label associated with the labelled pixels representing a cell type or biomarker of the cell types in each area of interest of the areas of interest; determining, by the at least one processor, a sum of values associated with each mask across all pixels; wherein the sum of the values represents a measure of an area of each area of interest of the areas of interest based at least in part on the image band of each area of interest; determining, by the at least one processor, a histopathological score based at least in part on the measure of the area of each area of interest; and causing to display, by the at least one processor, the histopathological score on at least one screen of at least one computing device associated with at least one user. ### Claim 2 (depends on 1) The method as recited in claim 1, wherein the histopathological scoring model comprises a generative adversarial network (GAN). ### Claim 3 (depends on 1) The method as recited in claim 1, wherein the labelled pixels comprises a set of image bands representing a set of classifications of cell types or biomarkers, wherein the at least one label of each mask comprises an image band of the set of image bands representing a classification by cell type or biomarker of a set of classifications by cell type or biomarker. ### Claim 4 (depends on 3) The method as recited in claim 3, wherein the set of image bands comprises a plurality of grayscale bands. ### Claim 5 (depends on 1) The method as recited in claim 1, further comprising normalizing, by the at least one processor, each mask to produce a pixel value for each pixel that is equivalent for every pixel in each mask. ### Claim 6 (depends on 5) The method as recited in claim 5, wherein the pixel value comprises a value of one. ### Claim 7 (depends on 1) The method as recited in claim 1, further comprising determining, by the at least one processor, a cell-type-specific histopathological score for a particular cell type or biomarker of the cell types or biomarkers based at least in part on a ratio of an area of a mask associated with the particular cell type or biomarker to a total area of the at least one mask. ### Claim 8 A system comprising: at least one processor in communication with at least one memory and configured to access instructions stored in the memory, wherein the instructions cause the at least one processor to perform steps to: receive a tissue image comprising a digital representation of a plurality of cells of a tissue; utilize a histopathological scoring model to predict at least one mask delineating areas of interest in the tissue image according to cell staining based on learned histopathological scoring parameters; wherein each mask of the at least one mask comprises: i) at least one polygon formed by labelled pixels representing a delineation of each area of interest of the areas of interest, and ii) at least one label associated with the labelled pixels representing an image band of each area of interest of the areas of interest; determine a sum of values associated with each image band across all pixels; wherein the sum of the values represents a measure of an area of each area of interest of the areas of interest based at least in part on the image band of each area of interest; determine a histopathological score based at least in part on the measure of the area of each area of interest; and cause to display the histopathological score on at least one screen of at least one computing device associated with at least one user. ### Claim 9 (depends on 8) The system as recited in claim 8, wherein the histopathological scoring model comprises a generative adversarial network (GAN). ### Claim 10 (depends on 8) The system as recited in claim 8, wherein the labelled pixels comprises a set of image bands representing a set of classifications of cell types or biomarkers, wherein the at least one label of each mask comprises an image band of the set of image bands representing a classification by cell type or biomarker of a set of classifications by cell type or biomarker. ### Claim 11 (depends on 10) The system as recited in claim 10, wherein the set of image bands comprises a plurality of grayscale bands. ### Claim 12 (depends on 8) The system as recited in claim 8, wherein the instructions further cause the at least one processor to perform steps to: normalize each mask to produce a pixel value for each pixel that is equivalent for every pixel in each mask. ### Claim 13 (depends on 12) The system as recited in claim 12, wherein the pixel value comprises a value of one. ### Claim 14 (depends on 8) The system as recited in claim 8, wherein the instructions further cause the at least one processor to perform steps to: determine a cell-type-specific histopathological score for a particular cell type or biomarker of the cell types based at least in part on a ratio of an area of a mask associated with the particular cell type or biomarker to a total area of the at least one mask. ### Claim 15 A non-transitory computer readable medium having software instructions stored thereon, the software instructions configured to cause at least one processor to perform steps comprising: receiving a tissue image comprising a digital representation of a plurality of cells of a tissue; utilizing a histopathological scoring model to predict at least one mask delineating areas of interest in the tissue image according to cell types or biomarkers based on learned histopathological scoring parameters; wherein each mask of the at least one mask comprises: i) at least one polygon formed by labelled pixels representing a delineation of each area of interest of the areas of interest, and ii) at least one label associated with the labelled pixels representing a cell type or biomarker of the cell types in each area of interest of the areas of interest; determining a sum of values associated with each mask across all pixels; wherein the sum of the values represents a measure of an area of each area of interest of the areas of interest based at least in part on the image band of each area of interest; determining a histopathological score based at least in part on the measure of the area of each area of interest; and causing to display the histopathological score on at least one screen of at least one computing device associated with at least one user. ### Claim 16 (depends on 15) The non-transitory computer readable medium as recited in claim 15, wherein the histopathological scoring model comprises a generative adversarial network (GAN). ### Claim 17 (depends on 15) The non-transitory computer readable medium as recited in claim 15, wherein the labelled pixels comprises a set of image bands representing a set of classifications of cell types or biomarkers, wherein the at least one label of each mask comprises an image band of the set of image bands representing a classification by cell type or biomarker of a set of classifications by cell type or biomarker. ### Claim 18 (depends on 17) The non-transitory computer readable medium as recited in claim 17, wherein the set of image bands comprises a plurality of grayscale bands. ### Claim 19 (depends on 15) The non-transitory computer readable medium as recited in claim 15, wherein the software instructions are further configured to cause at least one processor to perform steps comprising normalizing each mask to produce a pixel value for each pixel that is equivalent for every pixel in each mask. ### Claim 20 (depends on 15) The non-transitory computer readable medium as recited in claim 15, wherein the software instructions are further configured to cause the at least one processor to perform steps comprising determining a cell-type-specific histopathological score for a particular cell type or biomarker of the cell types based at least in part on a ratio of an area of a mask associated with the particular cell type or biomarker to a total area of the at least one mask.