# US 2014/0210973 A1 · Image processing device, endoscope system, image processing method, and computer-readable storage device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0210973 A1 - Document ID: US20140210973A1 - Status: Published application - Kind: A1 - Application number: 14/226,597 - Filed: 2014-03-26 - Published: 2014-07-31 - Earliest claimed: 2011-10-04 - Assignees: Olympus Corporation - Applicants: Olympus Corporation - Inventors: Jumpei TAKAHASHI - Web page: https://patentgrep.com/patent/US20140210973A1 ## CPC - G06T 5/70: Denoising; Smoothing - A61B 1/000095: {for image enhancement} - G06T 7/254: involving subtraction of images - A61B 1/00186: {with imaging filters} - G06T 2207/10016: Video; Image sequence - G06T 2207/10024: Color image - G06T 2207/10068: Endoscopic image - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) ## Abstract An image processing device includes an evaluation value calculation section that calculates an evaluation value that is used to determine whether or not an inter-frame state of an object within a captured image is a stationary state, an estimated noise amount acquisition section that acquires an estimated noise amount of the captured image, a determination section that determines whether or not the inter-frame state of the object is the stationary state based on the evaluation value and the estimated noise amount, and a noise reduction processing section that performs a first noise reduction process (time-direction noise reduction process) when it has been determined that the inter-frame state of the object is the stationary state, and performs a second noise reduction process that includes at least a spatial-direction noise reduction process when it has been determined that the inter-frame state of the object is not the stationary state. ## Claims ### Claim 1 An image processing device comprising: an evaluation value calculation section that calculates an evaluation value that is used to determine whether or not an inter-frame state of an object within a captured image is a stationary state; an estimated noise amount acquisition section that acquires an estimated noise amount of the captured image; a determination section that determines whether or not the inter-frame state of the object is the stationary state based on the evaluation value, the estimated noise amount, and a specific condition; and a noise reduction processing section that performs a first noise reduction process that is a time-direction noise reduction process on the captured image when it has been determined that the inter-frame state of the object is the stationary state, and performs a second noise reduction process that includes at least a spatial-direction noise reduction process on the captured image when it has been determined that the inter-frame state of the object is not the stationary state, the specific condition being an observation state of the object, or whether a target area of a noise reduction process belongs to a front-field-of-view area or a side-field-of-view area, or a type of the captured image. ### Claim 2 (depends on 1) The image processing device as defined in claim 1, further comprising: an angle-of-view information acquisition section that acquires angle-of-view information about an imaging section that captures the captured image, the determination section setting a determination condition for determining whether or not the inter-frame state of the object is the stationary state corresponding to an angle of view indicated by the angle-of-view information. ### Claim 3 (depends on 2) The image processing device as defined in claim 2, the evaluation value calculation section calculating an inter-frame difference value of the captured image as the evaluation value, and the determination section determining that the inter-frame state of the object is the stationary state when the angle of view is equal to or larger than a threshold value, and the inter-frame difference value is equal to or smaller than the estimated noise amount, and determining that the inter-frame state of the object is the stationary state when the angle of view is smaller than the threshold value, and the inter-frame difference value is equal to or smaller than a value obtained by multiplying the estimated noise amount by a coefficient that is larger than 1. ### Claim 4 (depends on 1) The image processing device as defined in claim 1, the determination section setting a determination condition for determining whether or not the inter-frame state of the object is the stationary state corresponding to an area of the captured image to which the target area of the noise reduction process belongs. ### Claim 5 (depends on 4) The image processing device as defined in claim 4, further comprising: an area determination section, an imaging section that captures the captured image being capable of capturing a front field of view and a side field of view of the imaging section, the area determination section determining whether the target area belongs to the front-field-of-view area of the captured image or the side-field-of-view area of the captured image, the front-field-of-view area being an area that corresponds to the front field of view, and the side-field-of-view area being an area that corresponds to the side field of view, and the determination section setting the determination condition that differs between a case where it has been determined that the target area belongs to the front-field-of-view area and a case where it has been determined that the target area belongs to the side-field-of-view area. ### Claim 6 (depends on 5) The image processing device as defined in claim 5, the evaluation value calculation section calculating an inter-frame difference value of the captured image as the evaluation value, and the determination section determining that the inter-frame state of the object is the stationary state when it has been determined that the target area belongs to the side-field-of-view area, and the inter-frame difference value is equal to or smaller than the estimated noise amount, and determining that the inter-frame state of the object is the stationary state when it has been determined that the target area belongs to the front-field-of-view area, and the inter-frame difference value is equal to or smaller than a value obtained by multiplying the estimated noise amount by a coefficient that is larger than 1. ### Claim 7 (depends on 5) The image processing device as defined in claim 5, the area determination section determining an area to which the target area belongs based on a position of the target area within the captured image. ### Claim 8 (depends on 1) The image processing device as defined in claim 1, the determination section setting a determination condition for determining whether or not the inter-frame state of the object is the stationary state corresponding to a type of the captured image. ### Claim 9 (depends on 8) The image processing device as defined in claim 8, the captured image being a white light image that includes information within a white wavelength band, or a special light image that includes information within a specific wavelength band, and the determination section setting the determination condition that differs between a case where the captured image is the white light image and a case where the captured image is the special light image. ### Claim 10 (depends on 9) The image processing device as defined in claim 9, the evaluation value calculation section calculating an inter-frame difference value of the captured image as the evaluation value, and the determination section determining that the inter-frame state of the object is the stationary state when the captured image is the white light image, and the inter-frame difference value is equal to or smaller than the estimated noise amount, and determining that the inter-frame state of the object is the stationary state when the captured image is the special light image, and the inter-frame difference value is equal to or smaller than a value obtained by multiplying the estimated noise amount by a coefficient that is smaller than 1. ### Claim 11 (depends on 9) The image processing device as defined in claim 9, the specific wavelength band being narrower than the white wavelength band. ### Claim 12 (depends on 11) The image processing device as defined in claim 11, the white light image and the special light image each being an in vivo image, and the specific wavelength band included in the in vivo image being a wavelength band of light absorbed by hemoglobin in blood. ### Claim 13 (depends on 12) The image processing device as defined in claim 12, the specific wavelength band being 390 to 445 nm or 530 to 550 nm. ### Claim 14 (depends on 1) The image processing device as defined in claim 1, further comprising: a motion vector detection section that detects a motion vector of the object within the captured image between a first frame and a second frame, the evaluation value calculation section calculating the evaluation value that is used to determine whether or not the inter-frame state of the object is the stationary state between the first frame and the second frame that is subjected to motion compensation using the motion vector, and the noise reduction processing section performing the first noise reduction process and the second noise reduction process based on the first frame and the second frame that is subjected to motion compensation using the motion vector. ### Claim 15 (depends on 1) The image processing device as defined in claim 1, the evaluation value calculation section calculating an inter-frame difference value of the captured image as the evaluation value, and the determination section using the estimated noise amount as a threshold value, and determining that the inter-frame state of the object is the stationary state when the inter-frame difference value is equal to or smaller than the threshold value. ### Claim 16 (depends on 15) The image processing device as defined in claim 15, an area of the captured image in a first frame that is situated around a processing target pixel and has a given size being a first area, and an area of the captured image in a second frame that is situated around the processing target pixel and has the given size being a second area, the evaluation value calculation section calculating a plurality of difference values between a pixel value of the first area and a pixel value of the second area while sequentially shifting the second area by one pixel relative to the first area in a horizontal direction and a vertical direction, and outputting a minimum value among the plurality of difference values as the inter-frame difference value, and the determination section determining that the inter-frame state of the object is the stationary state when the minimum value is equal to or smaller than the threshold value. ### Claim 17 An endoscope system comprising: an imaging section that captures a captured image; an evaluation value calculation section that calculates an evaluation value that is used to determine whether or not an inter-frame state of an object within the captured image is a stationary state; an estimated noise amount acquisition section that acquires an estimated noise amount of the captured image; a determination section that determines whether or not the inter-frame state of the object is the stationary state based on the evaluation value, the estimated noise amount, and a specific condition; and a noise reduction processing section that performs a first noise reduction process that is a time-direction noise reduction process on the captured image when it has been determined that the inter-frame state of the object is the stationary state, and performs a second noise reduction process that includes at least a spatial-direction noise reduction process on the captured image when it has been determined that the inter-frame state of the object is not the stationary state, the specific condition being an observation state of the object, or whether a target area of a noise reduction process belongs to a front-field-of-view area or a side-field-of-view area, or a type of the captured image. ### Claim 18 An image processing method comprising: calculating an evaluation value that is used to determine whether or not an inter-frame state of an object within a captured image is a stationary state; acquiring an estimated noise amount of the captured image; determining whether or not the inter-frame state of the object is the stationary state based on the evaluation value, the estimated noise amount, and a specific condition, the specific condition being an observation state of the object, or whether a target area of a noise reduction process belongs to a front-field-of-view area or a side-field-of-view area, or a type of the captured image; and performing a first noise reduction process that is a time-direction noise reduction process on the captured image when it has been determined that the inter-frame state of the object is the stationary state, and performing a second noise reduction process that includes at least a spatial-direction noise reduction process on the captured image when it has been determined that the inter-frame state of the object is not the stationary state. ### Claim 19 A computer-readable storage device with an executable program stored thereon, wherein the program instructs a computer to perform steps of: calculating an evaluation value that is used to determine whether or not an inter-frame state of an object within a captured image is a stationary state; acquiring an estimated noise amount of the captured image; determining whether or not the inter-frame state of the object is the stationary state based on the evaluation value and the estimated noise amount; and performing a first noise reduction process that is a time-direction noise reduction process on the captured image when it has been determined that the inter-frame state of the object is the stationary state, and performing a second noise reduction process that includes at least a spatial-direction noise reduction process on the captured image when it has been determined that the inter-frame state of the object is not the stationary state. ### Claim 20 (depends on 1) The image processing device as defined in claim 1, the observation state being a state relating to an angle of view of an imaging section that captures the captured image. ### Claim 21 (depends on 1) The image processing device as defined in claim 1, the type of the captured image being determined by a number of pixels of the captured image, a resolution, an exposure time when the captured image was captured, a frame rate, a type of a connected imaging section, characteristics of illumination light, or characteristics of an optical filter used for imaging.