# US 9,519,128 B2 · Image processing apparatus, microscope system, and image processing method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,519,128 B2 - Document ID: US9519128B2 - Status: Granted - Kind: B2 - Application number: 14/478,058 - Filed: 2014-09-05 - Published: 2014-12-25 - Granted: 2016-12-13 - Earliest claimed: 2012-03-07 - Assignees: Olympus Corporation - Applicants: Olympus Corporation - Inventors: Yoshioki Kaneko - Web page: https://patentgrep.com/patent/US9519128B2 ## CPC - G02B 21/0076: {arrangements using fluorescence or luminescence} - A61B 5/0071: {by measuring fluorescence emission} - G01N 21/6458: {Fluorescence microscopy (fluorescence microscopes per se G02B21/0076 and G02B21/16)} - G02B 21/06: Means for illuminating specimens - G02B 21/16: adapted for ultraviolet illumination {; Fluorescence microscopes (G02B21/0076 takes precedence)} - G02B 21/365: {Control or image processing arrangements for digital or video microscopes (G02B21/361, G02B21/362 take precedence)} - G02B 21/088: {for both incident illumination and transillumination} - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G01N: INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES (measuring or testing apparatus or processes other than immunoassay, involving enzymes or microorganisms C12M, C12Q) ## Abstract An image processing apparatus includes an image acquisition unit that acquires image information representing a fluorescence observation image of a specimen stained with hematoxylin-eosin, a spectrum generation unit that generates a plurality of spectra each representing a wavelength distribution of fluorescence intensity in a plurality of pixels in the fluorescence observation image, a pixel extraction unit that extracts at least two pixel groups with a feature of a particular spectrum from the plurality of pixels, and an image generation unit that generates an image based on the extracted pixel groups. ## Claims ### Claim 1 An image processing apparatus comprising: an image acquisition unit that acquires image information representing a fluorescence observation image of a specimen stained with hematoxylin-eosin; a spectrum generation unit that generates a plurality of spectra each representing a wavelength distribution of fluorescence intensity in a plurality of pixels in the fluorescence observation image; a pixel extraction unit that extracts at least two pixel groups with a feature of a particular spectrum from the plurality of pixels, wherein the pixel extraction unit is configured to: extract the at least two pixel groups with reference to at least two fluorescence intensity spectra acquired respectively in advance with respect to at least two kinds of components in the specimen; extract pixel groups having the same peak wavelengths as those of the at least two fluorescence intensity spectra, as the at least two pixel groups, respectively; extract a pixel group having a peak wavelength of 560 nm as a pixel group representing an elastic fiber in the specimen; and extract a pixel group having a peak wavelength of 570 nm as a pixel group representing a blood corpuscle in the specimen; an image generation unit that generates an image based on the extracted pixel groups; and an abnormality determination unit that determines a presence or absence of abnormality in a blood vessel in the fluorescence observation image based on the pixel group representing the elastic fiber. ### Claim 2 (depends on 1) The image processing apparatus according to claim 1, wherein the pixel extraction unit extracts, as the pixel group representing the elastic fiber, a pixel group whose fluorescence intensity at a peak wavelength is greater than or equal to a specified threshold from among a plurality of pixels in the pixel group having a peak wavelength of 560 nm. ### Claim 3 (depends on 1) The image processing apparatus according to claim 1, wherein the image acquisition unit further acquires second image information representing a bright-field observation image of the specimen stained with hematoxylin-eosin, and the image generation unit further generates a second image based on the second image information. ### Claim 4 (depends on 3) The image processing apparatus according to claim 3, further comprising an image display unit that displays at least one of the image and the second image. ### Claim 5 (depends on 3) The image processing apparatus according to claim 3, further comprising an image synthesis unit that generates a synthesis image obtained by overlapping the image on the second image. ### Claim 6 (depends on 5) The image processing apparatus according to claim 5, further comprising an image display unit that displays at least one of the image, the second image, and the synthesis image. ### Claim 7 (depends on 1) The image processing apparatus according to claim 1, wherein the abnormality determination unit includes: a blood vessel feature data calculation unit that calculates morphological feature data of a blood vessel based on the pixel group representing the elastic fiber; and a determination unit that determines the presence or absence of abnormality in the blood vessel based on the morphological feature data of the blood vessel. ### Claim 8 (depends on 1) The image processing apparatus according to claim 1, wherein the abnormality determination unit includes: an elastic fiber continuity determination unit that determines continuity of the elastic fiber; and a determination unit that determines the presence or absence of abnormality in the blood vessel based on a determination result of the continuity. ### Claim 9 (depends on 8) The image processing apparatus according to claim 8, wherein the elastic fiber continuity determination unit determines the continuity of the elastic fiber based on an area of a region surrounded by the elastic fiber and circularity of the elastic fiber. ### Claim 10 (depends on 1) The image processing apparatus according to claim 1, wherein the abnormality determination unit includes: a vascular wall thickness calculation unit that calculates a thickness of a vascular wall of the blood vessel based on the elastic fiber; and a determination unit that determines whether the blood vessel is normal or abnormal based on the thickness. ### Claim 11 (depends on 10) The image processing apparatus according to claim 10, wherein the vascular wall thickness calculation unit: measures an outer diameter of the blood vessel based on the elastic fiber; extracts a region inside the elastic fiber based on the elastic fiber; measures an inner diameter of the blood vessel based on the region inside the elastic fiber; and calculates the thickness of the vascular wall based on the outer diameter and the inner diameter of the blood vessel. ### Claim 12 (depends on 7) The image processing apparatus according to claim 7, further comprising a storage unit that stores a lookup table in which the morphological feature data of the blood vessel are correlated with a determination result of the presence or absence of abnormality and/or a degree of abnormality in the blood vessel, wherein the determination unit determines the presence or absence of abnormality and/or the degree of abnormality with reference to the lookup table. ### Claim 13 (depends on 8) The image processing apparatus according to claim 8, further comprising a storage unit that stores a lookup table in which the determination result of the continuity of the elastic fiber is correlated with a determination result of the presence or absence of abnormality and/or a degree of abnormality in the blood vessel, wherein the determination unit determines the presence or absence of abnormality and/or the degree of abnormality with reference to the lookup table. ### Claim 14 (depends on 10) The image processing apparatus according to claim 10, further comprising a storage unit that stores a lookup table in which an evaluation value related to the thickness of the vascular wall is correlated with a determination result of the presence or absence of abnormality and/or a degree of abnormality in the blood vessel, wherein the determination unit determines the presence or absence of abnormality and/or the degree of abnormality with reference to the lookup table. ### Claim 15 (depends on 1) A microscope system comprising: the image processing apparatus according to claim 1; a stage on which the specimen is configured to be placed; an epi-illumination optical system that emits excitation light toward the stage; an objective optical system that is provided to face the stage and receives light from a direction of the specimen; a filter that extracts fluorescence light from the light transmitted through the objective optical system; and an imaging unit that is provided on an optical path of the light transmitted through the objective optical system and generates image information by capturing an observation image of the specimen. ### Claim 16 (depends on 15) The microscope system according to claim 15, further comprising: a transmission illumination optical system that emits illumination light toward the stage; and a switching unit that switches between the excitation light and the illumination light to be emitted toward the specimen, wherein the filter is provided removably on the optical path of the light transmitted through the objective optical system and extracts the fluorescence light with a specified wavelength band from the light. ### Claim 17 (depends on 15) The microscope system according to claim 15, wherein a wavelength band of the excitation light is 300 nm or more and 400 nm or less. ### Claim 18 (depends on 17) The microscope system according to claim 17, wherein the wavelength band of the fluorescence light is 520 nm or more and 650 nm or less. ### Claim 19 (depends on 15) The microscope system according to claim 15, wherein the imaging unit includes a camera that is capable of capturing images in a plurality of wavelength bands. ### Claim 20 An image processing method comprising: acquiring image information representing a fluorescence observation image of a specimen stained with hematoxylin-eosin; generating a plurality of spectra each representing a wavelength distribution of fluorescence intensity in a plurality of pixels in the fluorescence observation image; extracting at least two pixel groups with a feature of a particular spectrum from the plurality of pixels, wherein the extracting: extracts the at least two pixel groups with reference to at least two fluorescence intensity spectra acquired respectively in advance with respect to at least two kinds of components in the specimen; extracts pixel groups having the same peak wavelengths as those of the at least two fluorescence intensity spectra, as the at least two pixel groups, respectively; extracts a pixel group having a peak wavelength of 560 nm as a pixel group representing an elastic fiber in the specimen; and extracts a pixel group having a peak wavelength of 570 nm as a pixel group representing a blood corpuscle in the specimen; generating an image based on the extracted pixel groups; and determining a presence or absence of abnormality in a blood vessel in the fluorescence observation image based on the pixel group representing the elastic fiber.