# US 10,502,728 B2 · Image processing device, pathological diagnosis support system, storage medium for image processing, and image processing method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,502,728 B2 - Document ID: US10502728B2 - Status: Granted - Kind: B2 - Application number: 16/207,378 - Filed: 2018-12-03 - Published: 2019-04-18 - Granted: 2019-12-10 - Earliest claimed: 2013-12-18 - Assignees: Konica Minolta, Inc. - Applicants: Konica Minolta, Inc. - Inventors: Yasuhiro Watanabe; Shuji Ichitani; Kohsuke Gonda; Noriaki Ohuchi; Mika Watanabe - Web page: https://patentgrep.com/patent/US10502728B2 ## CPC - G01N 33/4833: {of solid biological material, e.g. tissue samples, cell cultures (tissue in vivo A61B5/00; cell suspensions G01N33/48735)} - A61B 5/0071: {by measuring fluorescence emission} - G06T 5/50: using two or more images, e.g. averaging or subtraction - G06T 7/0012: {Biomedical image inspection} - G06T 7/11: Region-based segmentation - G06V 20/695: {Preprocessing, e.g. image segmentation} - G06T 2207/10064: Fluorescence image - G06T 2207/20221: Image fusion; Image merging - G06T 2207/30024: Cell structures in vitro; Tissue sections in vitro - G01N: INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES (measuring or testing apparatus or processes other than immunoassay, involving enzymes or microorganisms C12M, C12Q) - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract An image processing device (2A) comprises: an input means for inputting a brightfield image representing cell morphology in a tissue section, and a fluorescence image representing, by fluorescent bright spots, the expression of a specific protein in the same range of the tissue section; a first generation means for generating a cell image obtained by extracting a specific site of a cell from the brightfield image; a second generation means for generating an image obtained by extracting bright spot regions from the fluorescence image, creating a brightness profile for each bright spot region, and generating a fluorescent particle image obtained by extracting the fluorescent particles in the bright spot regions on the basis of the fluorescence profile for one fluorescent particle, which serves as a fluorescence bright spot source; and a calculation means for superimposing the cell image and the fluorescent particle image on one another. ## Claims ### Claim 1 An image processing device comprising: a processor including: an input unit to input, a bright field image showing a shape of a cell in a tissue section and a fluorescence image showing presence of a target biological substance as a fluorescent bright point in the same range of the tissue section as the bright field image; a first generation unit to generate a cell image showing an extracted specific site of the cell from the bright field image; a second generation unit to generate an image showing an extracted bright point region from the fluorescence image, to generate a luminance profile for the bright point region, to identify extracted fluorescent particles in the extracted bright point region using a reference luminance profile, and to generate a fluorescent particle image displaying the identified fluorescent particles in the bright point region; and an addition unit to add the cell image and the fluorescent particle image, thereby showing the identified fluorescent particles in each cell of the cell image. ### Claim 2 (depends on 1) The image processing device according to claim 1, wherein a source of fluorescent bright point includes two or more types of fluorescent particles having different emission wavelength from each other; and the second generation unit generates a fluorescent particle image for each type of the fluorescent particles. ### Claim 3 (depends on 1) The image processing device according to claim 1 wherein the target biological substance includes at least any one of a protein, a peptide, a nucleic acid, an oligonucleotide, a polynucleotide or an antibody. ### Claim 4 (depends on 1) A pathological diagnosis support system comprising: the image processing device according to claim 1; and an image acquisition device to acquire the bright field image and the fluorescence image used in the image processing device. ### Claim 5 An image processing method comprising; an input step of inputting to a processor a bright field image showing a shape of a cell in a tissue section and a fluorescence image showing presence of a target biological substance as a fluorescent bright point in the same range of the tissue section as the bright field image; a first generation step of generating, by the processor, a cell image showing an extracted specific site of the cell from the bright field image; a second generation step of generating an image showing an extracted bright point region from the fluorescence image, to generate a luminance profile for the bright point region, to identify extracted fluorescent particles in the extracted bright point region using a reference luminance profile of a single fluorescence particle, and to generate a fluorescent particle image showing the identified fluorescent particles in the bright point region; and an addition step of adding, by the processor, the cell image and the fluorescent particle image thereby displaying the identified fluorescent particles in each cell of the cell image. ### Claim 6 (depends on 5) The method of claim 5, further comprising providing a number and locations of the fluorescent particles within each bright point region on the basis of the luminance profile. ### Claim 7 (depends on 6) The method of claim 6, wherein the target biological substance includes at least any one of a protein, a peptide, a nucleic acid, an oligonucleotide, a polynucleotide or an antibody. ### Claim 8 (depends on 6) The method of claim 6, wherein a labeling is performed on the bright field image, thereby attaching a label to each of the extracted cell. ### Claim 9 (depends on 5) The method of claim 5, wherein a source of fluorescent bright point includes two or more types of fluorescent particles having different emission wavelengths from each other; and the second generation step includes a step of generating a fluorescent particle image for each type of the fluorescent particles. ### Claim 10 (depends on 9) The method of claim 9, further comprising providing information of a number and locations of the fluorescent particles within each bright point region on the basis of the luminance profile. ### Claim 11 A method for forming an image by using a bright field image to show a shape of a cell in a tissue section and a fluorescence image to show presence of a target biological substance as a fluorescent bright point in the same range of the tissue section as the bright field image, comprising; a first step of generating by a processor a cell image showing an extracted specific site of the cell from the bright field image; a second generation step of generating by the processor an image showing an extracted bright point region from the fluorescence image, to generate a luminance profile for the bright point region, to identify extracted fluorescent particles in the extracted bright point region using a reference luminance profile of a single fluorescence particle, and to generate a fluorescent particle image showing the identified fluorescent particles in the bright point region; and an addition step of adding by the processor the cell image and the fluorescent particle image thereby displaying the identified fluorescent particles in each cell of the cell image. ### Claim 12 (depends on 11) The method of claim 11, wherein a source of fluorescent bright point includes two or more types of fluorescent particles having different emission wavelengths from each other; and the second generation step includes a step of generating a fluorescent particle image for each type of the fluorescent particles.