# US 11,308,616 B2 · Systems and methods to process electronic images to provide image-based cell group targeting > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,308,616 B2 - Document ID: US11308616B2 - Status: Granted - Kind: B2 - Application number: 17/391,997 - Filed: 2021-08-02 - Published: 2022-02-10 - Granted: 2022-04-19 - Earliest claimed: 2020-08-04 - Assignees: Paige.AI, Inc. - Applicants: PAIGE.AI, Inc. - Inventors: Rodrigo Ceballos Lentini; Christopher Kanan; Belma Dogdas - Web page: https://patentgrep.com/patent/US11308616B2 ## CPC - G06T 7/0012: {Biomedical image inspection} - G06V 20/69: Microscopic objects, e.g. biological cells or cellular parts - G06F 18/23213: with fixed number of clusters, e.g. K-means clustering - G06N 20/00: Machine learning - G06T 7/11: Region-based segmentation - G06V 10/70: using pattern recognition or machine learning (optical pattern recognition or electronic computations therefor G06V10/88) - G06V 10/763: {Non-hierarchical techniques, e.g. based on statistics of modelling distributions} - G06V 30/19107: {Clustering techniques} - G16B 40/00: ICT specially adapted for biostatistics; ICT specially adapted for bioinformatics-related machine learning or data mining, e.g. knowledge discovery or pattern finding - G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems - G06T 2207/20081: Training; Learning - G06T 2207/30024: Cell structures in vitro; Tissue sections in vitro - G06T 2207/30096: Tumor; Lesion - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G16B: BIOINFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR GENETIC OR PROTEIN-RELATED DATA PROCESSING IN COMPUTATIONAL MOLECULAR BIOLOGY - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA ## Abstract Systems and methods are disclosed for grouping cells in a slide image that share a similar target, comprising receiving a digital pathology image corresponding to a tissue specimen, applying a trained machine learning system to the digital pathology image, the trained machine learning system being trained to predict at least one target difference across the tissue specimen, and determining, using the trained machine learning system, one or more predicted clusters, each of the predicted clusters corresponding to a subportion of the tissue specimen associated with a target. ## Claims ### Claim 1 A computer-implemented method for grouping cells in a slide image that share a similar target, the method comprising: receiving a digital pathology image corresponding to a tissue specimen; applying a trained machine learning system to the digital pathology image, the trained machine learning system being trained to predict at least one target difference across the tissue specimen; and determining, using the trained machine learning system, one or more predicted clusters, each of the predicted clusters corresponding to a subportion of the tissue specimen associated with a target. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, further comprising: mapping, using the trained machine learning system, the predicted clusters to at least one known biological target. ### Claim 3 (depends on 2) The computer-implemented method of claim 2, further comprising: receiving at least one genetic sequence of the tissue specimen, a location of the genetic sequence being based on the mapping. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, further comprising: using a spatial distribution of the predicted clusters to determine, by the trained machine learning system, a recommended treatment decision. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, wherein applying the trained machine learning system comprises using an artificial intelligence (AI)-predicted segmentation with a clustering heuristic to determine an optimal sampling location of the tissue specimen to maximize information gained about the target difference across the tissue specimen. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, further comprising: determining a target composition of a set of samples associated with the tissue specimen, the samples having been processed using an available sequencing technique; and using an AI-predicted segmentation to infer a spatial distribution over factors in a slide. ### Claim 7 (depends on 1) The computer-implemented method of claim 1, the trained machine learning system having been trained by: receiving a plurality of digital pathology training images, the training images containing or being associated with tissue metadata; and training the trained machine learning system to predict clusters based on the digital pathology training images and the tissue metadata. ### Claim 8 (depends on 7) The computer-implemented method of claim 7, wherein training the trained machine learning system further comprises: using at least one of supervised learning, weakly supervised learning, and/or unsupervised learning. ### Claim 9 (depends on 8) The computer-implemented method of claim 8, wherein using supervised learning comprises: inputting the plurality of digital pathology training images of tissues with a spatial transcriptomic and/or imaging mass cytometry techniques; and training the trained machine learning system for segmentation of predicting a ground truth spatial transcriptomic and/or genetic marker masks. ### Claim 10 (depends on 8) The computer-implemented method of claim 8, wherein using weakly supervised learning comprises: using knowledge about a presence or absence of at least one target in a slide; and applying Multiple Instance Learning to learn a segmentation or predicting the at least one target spatially. ### Claim 11 (depends on 8) The computer-implemented method of claim 8, wherein unsupervised learning comprises: using autoencoders to learn and generate high-level embeddings for small regions of slides that encode morphological differences; and using k-means and/or hierarchical clustering to generate groupings of cells with similar morphological embeddings. ### Claim 12 (depends on 1) The computer-implemented method of claim 1, further comprising using flow cytometry and mass spectrometry techniques to determine physical and chemical characteristics of the tissue specimen. ### Claim 13 (depends on 1) The computer-implemented method of claim 1, further comprising outputting an image identifying the predicted clusters as one or more pixel masks. ### Claim 14 (depends on 1) The computer-implemented method of claim 1, wherein the predicted clusters comprise a set of points corresponding to a center of a found antigen cluster and/or a pixel mask segmenting an image into one or more antigen clusters. ### Claim 15 (depends on 1) The computer-implemented method of claim 1, further comprising: determining predicted mappings using the predicted clusters, the predicted mappings comprising a map from any of at least one cluster, of the predicted clusters, to a predicted antigen, and for each of the predicted clusters, a plurality of likely antigens. ### Claim 16 A system for grouping cells in a slide image that share a similar target, comprising: at least one memory storing instructions; and at least one processor configured to execute the instructions to perform operations comprising: receiving a digital pathology image corresponding to a tissue specimen; applying a trained machine learning system to the digital pathology image, the trained machine learning system being trained to predict at least one target difference across the tissue specimen; and determining, using the trained machine learning system, one or more predicted clusters, each of the predicted clusters corresponding to a subportion of the tissue specimen associated with a target. ### Claim 17 (depends on 16) The system of claim 16, the operations further comprising: mapping, using the trained machine learning system, the predicted clusters to at least one known biological target. ### Claim 18 (depends on 17) The system of claim 17, the operations further comprising: receiving at least one genetic sequence of the tissue specimen, a location of the genetic sequence being based on the mapping. ### Claim 19 A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations for grouping cells in a slide image that share a similar target, the operations comprising: receiving a digital pathology image corresponding to a tissue specimen; applying a trained machine learning system to the digital pathology image, the trained machine learning system being trained to predict at least one target difference across the tissue specimen; and determining, using the trained machine learning system, one or more predicted clusters, each of the predicted clusters corresponding to a subportion of the tissue specimen associated with a target. ### Claim 20 (depends on 19) The non-transitory computer-readable medium of claim 19, the operations further comprising: mapping, using the trained machine learning system, the predicted clusters to at least one known biological target.