# US 2014/0153811 A1 · Systems and methods for using an immunostaining mask to selectively refine ish analysis results > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0153811 A1 - Document ID: US20140153811A1 - Status: Published application - Kind: A1 - Application number: 13/693,406 - Filed: 2012-12-04 - Published: 2014-06-05 - Earliest claimed: 2012-12-04 - Assignees: General Electric Company - Applicants: General Electric Company - Inventors: Antti Seppo; Yousef Al-Kofahi; Dirk R. Padfield - Web page: https://patentgrep.com/patent/US20140153811A1 ## CPC - G06T 7/0012: {Biomedical image inspection} - G06T 7/0014: {using an image reference approach} - G06V 20/695: {Preprocessing, e.g. image segmentation} - G06V 20/698: {Matching; Classification} - G06T 2207/10064: Fluorescence image - G06T 2207/30024: Cell structures in vitro; Tissue sections in vitro - G06T 2207/30072: Microarray; Biochip, DNA array; Well plate - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A computer-implemented method of processing image data representing biological units in a tissue sample includes receiving a first image of the tissue sample containing signals from an immunofluorescent (IF) morphological marker, wherein the tissue sample is stained with the IF morphological marker, and receiving a second image of the same tissue sample containing signals from a fluorescent probe, wherein the tissue sample is hybridized in situ with the fluorescent probe. The method further includes classifying each biological unit in the tissue sample into one of at least two classes based on a mean intensity of the signals from the IF morphological marker in the first image, performing a fluorescence in situ hybridization (FISH) analysis of the tissue sample in the second image to obtain results therefrom, and filtering the results of the FISH analysis to produce a subset of the results pertaining to biological units classified in one class. ## Claims ### Claim 1 A computer-implemented method of processing image data representing biological units in a tissue sample, the computer including a processor, the method comprising: receiving, by the processor, a first image of the tissue sample containing signals from an immunofluorescent (IF) morphological marker, wherein the tissue sample is stained with the IF morphological marker; receiving, by the processor, a second image of the same tissue sample containing signals from a fluorescent probe, wherein the tissue sample is hybridized in situ with the fluorescent probe; classifying, by the processor, each biological unit in the tissue sample into one of at least two classes based on a mean intensity of the signals from the IF morphological marker in the first image; performing, by the processor, a fluorescence in situ hybridization (FISH) analysis of the tissue sample in the second image to obtain results therefrom; and filtering, by the processor, the results of the FISH analysis to produce a subset of the results pertaining to biological units classified in one of the at least two classes only. ### Claim 2 (depends on 1) The method of claim 1, wherein the at least two classes include epithelial and non-epithelial. ### Claim 3 (depends on 1) The method of claim 1, further comprising registering, by the processor, locations of signals from the IF morphological marker in the first image with locations of signals from the fluorescent probe in the second image to produce a registered image. ### Claim 4 (depends on 3) The method of claim 3, further comprising segmenting, by the processor, the second image to identify nuclei in the tissue sample based on the locations of signals from the fluorescent probe in the second image. ### Claim 5 (depends on 4) The method of claim 4, wherein each biological unit corresponds to respective nuclei identified in the tissue sample. ### Claim 6 (depends on 4) The method of claim 4, wherein segmenting the nuclei includes applying, by the processor, a wavelet transform to the second image. ### Claim 7 (depends on 6) The method of claim 6, further comprising generating, by the processor, a Voronoi partition and rings generated from a threshold distance map from the nuclei in the second image in which each respective ring is constrained to at least partially surround only one nucleus in the tissue sample. ### Claim 8 (depends on 7) The method of claim 7, wherein generating the Voronoi partition in the second image enables generating local background regions from the second image using nuclei identified in the tissue sample as seeds of the Voronoi partition multiplied by a mask defined by each ring. ### Claim 9 (depends on 7) The method of claim 7, wherein the IF morphological marker is configured to target a cytokeratin (CK) protein, and wherein the method further comprises thresholding the first image using a threshold value in conjunction with Otsu's thresholding method and estimating a minimum intensity level of an epithelial region in the first image. ### Claim 10 (depends on 9) The method of claim 9, further comprising changing the threshold value interactively by a user while observing a change in classification of each biological unit. ### Claim 11 (depends on 9) The method of claim 9, wherein each nucleus is classified as one of epithelial and non-epithelial by computing a mean cytokeratin (CK) intensity within the ring surrounding the nucleus and comparing the mean CK intensity to the estimated minimum intensity level of the epithelial region. ### Claim 12 (depends on 1) The method of claim 1, wherein the IF morphological marker includes a 4′,6-diamidino-2-phenylindole (DAPI) stain. ### Claim 13 (depends on 1) The method of claim 1, wherein the IF morphological marker is configured to target at least one of a cytokeratin (CK) protein and a Human Epidermal Growth Factor Receptor 2 (HER2) protein. ### Claim 14 (depends on 1) The method of claim 1, wherein the fluorescent probe is configured to bind to least one of a HER2 gene and a Chromosome 17 centrometric repeat. ### Claim 15 A non-transitory computer-readable medium having stored thereon computer-executable instructions that when executed by a computer cause the computer to: receive a first image of a tissue sample containing signals from an immunofluorescent (IF) morphological marker, wherein the tissue sample is stained with the IF morphological marker; receive a second image of the same tissue sample containing signals from a fluorescent probe, wherein the tissue sample is hybridized in situ with the fluorescent probe; classify each biological unit in the tissue sample into one of at least two classes based on a mean intensity of the signals from the IF morphological marker in the first image; perform a fluorescence in situ hybridization (FISH) analysis of the tissue sample in the second image to obtain results therefrom; and filter the results of the FISH analysis to produce a subset of the results pertaining to biological units classified as epithelial only. ### Claim 16 (depends on 15) The computer readable medium of claim 15, further comprising computer-executable instructions that when executed by the computer cause the computer to: register locations of signals from the IF morphological marker in the first image with locations of signals from the fluorescent probe in the second image to produce a registered image. ### Claim 17 (depends on 16) The computer readable medium of claim 16, further comprising computer-executable instructions that when executed by the computer cause the computer to: segment the second image to identify nuclei in the tissue sample based on the locations of signals from the fluorescent probe in the second image. ### Claim 18 A system for processing image data representing biological units in a tissue sample, the system comprising: a processor; an input in electrical communication with the processor and configured to receive the image data; and a memory in electrical communication with the processor, the memory including computer-executable instructions that when executed by the processor cause the processor to: receive a first image of a tissue sample containing signals from an immunofluorescent (IF) morphological marker, wherein the tissue sample is stained with the IF morphological marker; receive a second image of the same tissue sample containing signals from a fluorescent probe, wherein the tissue sample is hybridized in situ with the fluorescent probe; classify each biological unit in the tissue sample into one of at least two classes based on a mean intensity of the signals from the IF morphological marker in the first image; perform a fluorescence in situ hybridization (FISH) analysis of the tissue sample in the second image to obtain results therefrom; and filter the results of the FISH analysis to produce a subset of the results pertaining to biological units classified in one of the at least two classes only. ### Claim 19 (depends on 18) The system of claim 18, wherein the at least two classes include epithelial and non-epithelial. ### Claim 20 (depends on 18) The system of claim 18, wherein the memory further includes computer-executable instructions that when executed by the processor cause the processor to: register locations of signals from the IF morphological marker in the first image with locations of signals from the fluorescent probe in the second image to produce a registered image. ### Claim 21 (depends on 20) The system of claim 20, wherein the memory further includes computer-executable instructions that when executed by the processor cause the processor to: segment the second image to identify nuclei in the tissue sample based on the locations of signals from the fluorescent probe in the second image. ### Claim 22 (depends on 21) The system of claim 21, wherein each biological unit corresponds to respective nuclei identified in the tissue sample. ### Claim 23 (depends on 21) The system of claim 21, wherein the memory further includes computer-executable instructions that when executed by the processor cause the processor to apply a wavelet transform to the second image to segment the nuclei in the tissue sample. ### Claim 24 (depends on 23) The system of claim 23, wherein the memory further includes computer-executable instructions that when executed by the processor cause the processor to generate a Voronoi partition and rings generated from a threshold distance map from the nuclei in the second image in which each respective ring is constrained to at least partially surround only one nucleus in the tissue sample. ### Claim 25 (depends on 23) The system of claim 23, wherein the memory further includes computer-executable instructions that when executed by the processor cause the processor to generate local background regions from the second image using nuclei identified in the tissue sample as seeds of the Voronoi partition multiplied by a mask defined by each ring. ### Claim 26 (depends on 18) The system of claim 18, wherein the IF morphological marker includes a 4′,6-diamidino-2-phenylindole (DAPI) stain. ### Claim 27 (depends on 18) The system of claim 18, wherein the IF morphological marker is configured to target at least one of a cytokeratin (CK) protein and a Human Epidermal Growth Factor Receptor 2 (HER2) protein. ### Claim 28 (depends on 18) The system of claim 18, wherein the fluorescent probe is configured to bind to least one of a HER2 gene and a Chromosome 17 centrometric repeat.