US 9,754,375 B2 · Image processing apparatus, image processing method, and non-transitory storage medium storing image processing program

Abstract

A similarity calculation unit calculates a similarity between pixel values of a target pixel and a reference pixel and a similarity between the pixel value of the target pixel and an average value of pixel values of reference pixels, in an input image. A weighting factor calculation unit calculates weighting factors based on the calculated similarities. A weighted average value calculation unit calculates weighted average values of the pixel value of the reference pixel and the average value of the pixel values of the reference pixels by use of the calculated weighting factors. A subtraction unit calculates a difference value between the pixel values of the target pixel and the reference pixel. An isolation degree calculation unit calculates an isolation degree based on the calculated deference value. A correction unit corrects the target pixel based on the calculated isolation degree.

Drawings

Drawings are temporarily unavailable. Open the PDF at USPTO

Drawings

Claims13

Dependent claims

An apparatus comprising:

a memory storing a program;
and
a processor which executes the program stored in the memory to perform operations including:
calculating a similarity between a pixel value of a target pixel and a pixel value of a reference pixel located in a vicinity of the target pixel and a similarity between the pixel value of the target pixel and an average value of pixel values of reference pixels located in the vicinity of the target pixel, in an input image;
calculating a weighting factor respectively for the pixel value of the reference pixel and the average value of the pixel values of the reference pixels having such a similarity that a ratio or a difference with respect to a multiplication result of multiplying a highest similarity among the calculated similarities by a coefficient equal to or more than a first predetermined threshold is equal to or more than a second predetermined threshold, the weighting factor being higher when the ratio is lower or the difference is smaller;
calculating a weighted average value of the pixel value of the reference pixel and the average value of the pixel values of the reference pixels by use of the calculated weighting factors;
calculating a difference value between the pixel value of the target pixel and the weighted average value;
calculating an isolation degree to judge whether the target pixel is a pixel to be corrected on the basis of the calculated difference value;
and
correcting the target pixel on the basis of the calculated isolation degree.

7 more elements

An apparatus comprising:

a memory storing a program;
and
a processor which executes the program stored in the memory to perform operations including:
generating a target pixel feature vector having, as a component, a pixel value of a pixel included in a target pixel peripheral region which is a region including pixels on a periphery of a target pixel in an input image;
for each of a plurality of reference pixels, comparing, pixel by pixel, color information included in the target pixel peripheral region with color information included in a reference pixel peripheral region which is a region including pixels on a periphery of a reference pixel located in the vicinity of the target pixel, and generating (i) when the color information included in the target pixel peripheral region is totally equal to the color information included in the reference pixel peripheral region, a reference pixel feature vector having, as a component, a pixel value included in the reference pixel peripheral region, or (ii) when there is a pixel in which the color information included in the target pixel peripheral region is different from the color information included in the reference pixel peripheral region, a reference pixel feature vector having, as a component, a pixel value in which a pixel value of the pixel included in the reference pixel peripheral region and having color information different from color information of the target pixel peripheral region is replaced with a pixel value calculated by use of a pixel which is a pixel located on the periphery of a pixel included in the reference pixel peripheral region and having color information different from color information of the target pixel peripheral region and which has color information equal to color information of the target pixel peripheral region;
calculating a similarity between the generated target pixel feature vector and each of the generated reference pixel feature vectors;
calculating a weighting factor respectively for the pixel value of each of the reference pixels having such a similarity that a ratio or a difference with respect to a multiplication result of multiplying a highest similarity among the calculated similarities by a coefficient equal to or more than a first predetermined threshold is equal to or more than a second predetermined threshold, whereby the weighting factor is higher when the ratio is lower or the difference is smaller;
calculating a weighted average value of the pixel values of the reference pixels by use of the weighting factors;
calculating a difference value between the pixel value of the target pixel and the weighted average value;
calculating an isolation degree to judge whether the target pixel is a pixel to be corrected on the basis of the calculated difference value;
and
correcting the target pixel on the basis of the calculated isolation degree.

9 more elements

An apparatus comprising:

a memory storing a program;
and
a processor which executes the program stored in the memory to perform operations including:
detecting an isolation degree of each pixel in an input image;
generating a target pixel feature vector having, as a component, a pixel value of a target pixel peripheral region which is a region including pixels on a periphery of a target pixel in the input image;
generating a target isolation degree feature vector having, as a component, an isolation degree included in the target pixel peripheral region for the isolation degree of the target pixel;
for each of a plurality of reference pixels, comparing, pixel by pixel, color information included in the target pixel peripheral region with color information included in a reference pixel peripheral region which is a region including pixels on a periphery of a reference pixel located in the vicinity of the target pixel, and generating (i) when the color information included in the target pixel peripheral region is totally equal to the color information included in the reference pixel peripheral region, a reference pixel feature vector having, as a component, a pixel value included in the reference pixel peripheral region and a reference isolation degree feature vector having, as a component, an isolation degree included in the reference pixel peripheral region, or (ii) when there is a pixel in which the color information included in the target pixel peripheral region is different from the color information included in the reference pixel peripheral region, reference pixel feature vector having, as a component, a pixel value in which a pixel value of the pixel included in the reference pixel peripheral region and having color information different from color information of the target pixel peripheral region is replaced with a pixel value calculated by use of a pixel which is a pixel located on the periphery of a pixel included in the reference pixel peripheral region and having color information different from color information of the target pixel peripheral region and which has color information equal to color information of the target pixel peripheral region, and a reference isolation degree feature vector having, as a component, an isolation degree in which the isolation degree of the pixel included in the reference pixel peripheral region and having color information different from color information of the target pixel peripheral region is replaced with an isolation degree calculated by use of an isolation degree of a pixel which is a pixel located on the periphery of a pixel included in the reference pixel peripheral region and having color information different from color information of the target pixel peripheral region and which has color information equal to color information of the target pixel peripheral region;
calculating a similarity between the target pixel feature vector and each of the generated reference pixel feature vectors by use of the target isolation degree feature vector and the reference isolation degree feature vectors;
calculating a weighting factor of each of the reference pixels respectively, on the basis of the calculated similarity and the detected isolation degree;
calculating, as a correction value, a weighted average value of the reference pixels by use of the calculated weighting factors;
and
mixing a pixel value of the target pixel with the correction value on the basis of the detected isolation degree.

9 more elements

An image processing method comprising:

generating a target pixel feature vector having, as a component, a pixel value of a pixel included in a target pixel peripheral region which is a region including pixels on a periphery of a target pixel in an input image;
for each of a plurality of reference pixels, comparing, pixel by pixel, color information included in the target pixel peripheral region with color information included in a reference pixel peripheral region which is a region including pixels on a periphery of a reference pixel located in the vicinity of the target pixel, and generating (i) when the color information included in the target pixel peripheral region is totally equal to the color information included in the reference pixel peripheral region, a reference pixel feature vector having, as a component, a pixel value included in the reference pixel peripheral region, or (ii) when there is a pixel in which the color information included in the target pixel peripheral region is different from the color information included in the reference pixel peripheral region, a reference pixel feature vector having, as a component, a pixel value in which a pixel value of the pixel included in the reference pixel peripheral region and having color information different from color information of the target pixel peripheral region is replaced with a pixel value calculated by use of a pixel which is a pixel located on the periphery of a pixel included in the reference pixel peripheral region and having color information different from color information of the target pixel peripheral region and which has color information equal to color information of the target pixel peripheral region;
calculating a similarity between the generated target pixel feature vector and each of the generated reference pixel feature vectors;
calculating a weighting factor respectively for the pixel value of each of the reference pixels having such a similarity that a ratio or a difference with respect to a multiplication result of multiplying a highest similarity among the calculated similarities by a coefficient equal to or more than a first predetermined threshold is equal to or more than a second predetermined threshold, whereby the weighting factor is higher when the ratio is lower or the difference is smaller;
calculating a weighted average value of the pixel values of the reference pixels by use of the weighting factors;
calculating a difference value between the pixel value of the target pixel and the weighted average value;
calculating an isolation degree to judge whether the target pixel is a pixel to be corrected on the basis of the calculated difference value;
and
correcting the target pixel on the basis of the calculated isolation degree.

6 more elements

An image processing method comprising:

detecting an isolation degree of each pixel in an input image;
generating a target pixel feature vector having, as a component, a pixel value of a target pixel peripheral region which is a region including pixels on the periphery of a target pixel in the input image;
generating a target isolation degree feature vector having, as a component, an isolation degree included in the target pixel peripheral region for the isolation degree of the target pixel;
for each of a plurality of reference pixels, comparing, pixel by pixel, color information included in the target pixel peripheral region with color information included in a reference pixel peripheral region which is a region including pixels on the periphery of a reference pixel located in the vicinity of the target pixel, and generating (i) when the color information included in the target pixel peripheral region is totally equal to the color information included in the reference pixel peripheral region, a reference pixel feature vector having, as a component, a pixel value included in the reference pixel peripheral region and a reference isolation degree feature vector having, as a component, an isolation degree included in the reference pixel peripheral region, or (ii) when there is a pixel in which the color information included in the target pixel peripheral region is different from the color information included in the reference pixel peripheral region, a reference pixel feature vector having, as a component, a pixel value in which a pixel value of the pixel included in the reference pixel peripheral region and having color information different from color information of the target pixel peripheral region is replaced with a pixel value calculated by use of a pixel which is a pixel located on the periphery of a pixel included in the reference pixel peripheral region and having color information different from color information of the target pixel peripheral region and which has color information equal to color information of the target pixel peripheral region, and a reference isolation degree feature vector having, as a component, an isolation degree in which the isolation degree of the pixel included in the reference pixel peripheral region and having color information different from color information of the target pixel peripheral region is replaced with an isolation degree calculated by use of an isolation degree of a pixel which is a pixel located on the periphery of a pixel included in the reference pixel peripheral region and having color information different from color information of the target pixel peripheral region and which has color information equal to color information of the target pixel peripheral region;
calculating a similarity between the target pixel feature vector and each of the generated reference pixel feature vectors by use of the target isolation degree feature vector and the reference isolation degree feature vectors;
calculating a weighting factor of each of the reference pixels respectively, on the basis of the calculated similarity and the detected isolation degree;
calculating, as a correction value, a weighted average value of the reference pixels by use of the calculated weighting factors;
and
mixing a pixel value of the target pixel with the correction value on the basis of the detected isolation degree.

6 more elements

A non-transitory storage medium storing an image processing program which causes a computer to perform operations including:

generate a target pixel feature vector having, as a component, a pixel value of a pixel included in a target pixel peripheral region which is a region including pixels on a periphery of a target pixel in an input image;
for each of a plurality of reference pixels, comparing, pixel by pixel, color information included in the target pixel peripheral region with color information included in a reference pixel peripheral region which is a region including pixels on a periphery of a reference pixel located in the vicinity of the target pixel, and generating (i) when the color information included in the target pixel peripheral region is totally equal to the color information included in the reference pixel peripheral region, a reference pixel feature vector having, as a component, a pixel value included in the reference pixel peripheral region, or (ii) when there is a pixel in which the color information included in the target pixel peripheral region is different from the color information included in the reference pixel peripheral region, a reference pixel feature vector having, as a component, a pixel value in which a pixel value of the pixel included in the reference pixel peripheral region and having color information different from color information of the target pixel peripheral region is replaced with a pixel value calculated by use of a pixel which is a pixel located on the periphery of a pixel included in the reference pixel peripheral region and having color information different from color information of the target pixel peripheral region and which has color information equal to color information of the target pixel peripheral region;
calculating a similarity between the generated target pixel feature vector and each of the generated reference pixel feature vectors;
calculating a weighting factor respectively for the pixel value of each of the reference pixels having such a similarity that a ratio or a difference with respect to a multiplication result of multiplying a highest similarity among the calculated similarities by a coefficient equal to or more than a first predetermined threshold is equal to or more than a second predetermined threshold, whereby the weighting factor is higher when the ratio is lower or the difference is smaller;
calculating a weighted average value of the pixel values of the reference pixels by use of the weighting factors;
calculating a difference value between the pixel value of the target pixel and the weighted average value;
calculating an isolation degree to judge whether the target pixel is a pixel to be corrected on the basis of the calculated difference value;
and
correcting the target pixel on the basis of the calculated isolation degree.

6 more elements

A non-transitory storage medium storing an image processing program which causes a computer to perform operations including:

detecting an isolation degree of each pixel in an input image;
generating a target pixel feature vector having, as a component, a pixel value of a target pixel peripheral region which is a region including pixels on a periphery of a target pixel in the input image;
generating a target isolation degree feature vector having, as a component, an isolation degree included in the target pixel peripheral region for the isolation degree of the target pixel;
for each of a plurality of reference pixels, comparing, pixel by pixel, color information included in the target pixel peripheral region with color information included in a reference pixel peripheral region which is a region including pixels on a periphery of a reference pixel located in the vicinity of the target pixel, and generating (i) when the color information included in the target pixel peripheral region is totally equal to the color information included in the reference pixel peripheral region, a reference pixel feature vector having, as a component, a pixel value included in the reference pixel peripheral region and a reference isolation degree feature vector having, as a component, an isolation degree included in the reference pixel peripheral region, or (ii) when there is a pixel in which the color information included in the target pixel peripheral region is different from the color information included in the reference pixel peripheral region, generate a reference pixel feature vector having, as a component, a pixel value in which a pixel value of the pixel included in the reference pixel peripheral region and having color information different from color information of the target pixel peripheral region is replaced with a pixel value calculated by use of a pixel which is a pixel located on the periphery of a pixel included in the reference pixel peripheral region and having color information different from color information of the target pixel peripheral region and which has color information equal to color information of the target pixel peripheral region, and a reference isolation degree feature vector having, as a component, an isolation degree in which the isolation degree of the pixel included in the reference pixel peripheral region and having color information different from color information of the target pixel peripheral region is replaced with an isolation degree calculated by use of an isolation degree of a pixel which is a pixel located on the periphery of a pixel included in the reference pixel peripheral region and having color information different from color information of the target pixel peripheral region and which has color information equal to color information of the target pixel peripheral region;
calculating a similarity between the target pixel feature vector and each of the generated reference pixel feature vectors by use of the target isolation degree feature vector and the reference isolation degree feature vectors;
calculating a weighting factor of each of the reference pixels respectively, on the basis of the calculated similarity and the detected isolation degree;
calculating, as a correction value, a weighted average value of the reference pixels by use of the calculated weighting factors;
and
mixing a pixel value of the target pixel with the correction value on the basis of the detected isolation degree.

6 more elements

Dependencies are inferred from the claim text.

Description13,070 words

Summary4,267 words

#1. Field of the Invention

#The present invention relates to an image processing apparatus, an image processing method, and a non-transitory storage medium storing an image processing program for detecting and correcting pixels to be corrected such as defective pixels and pixels of impulse noise from an image.

#2. Description of the Related Art

Brief description of the drawings21 figures

#The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention, and together with the general description given above and the detailed description of the embodiments given below, serve to explain the principles of the invention.

#FIG. 1 is a diagram showing a configuration example of an information processing apparatus including an image processing apparatus according to each embodiment of the present invention;

#FIG. 2 is a block diagram showing a configuration example of an image processor as the image processing apparatus according to the first embodiment of the present invention;

Detailed description8,265 words

#Hereinafter, embodiments of the present invention will be described with reference to the drawings.

#FIG. 1 is a diagram showing a configuration example of an information processing apparatus including an image processing apparatus according to each embodiment of the present invention. An information processing apparatus 1 shown in FIG. 1 is, for example, a personal computer (PC). However, the information processing apparatus 1 does not always need to be a PC, and may include various types of information processing apparatuses having information processing functions. For example, the information processing apparatus 1 may be a digital camera.

#The information processing apparatus 1 includes a controller 2, storage medium 3, an image processor 4, a display 5, an operation unit 6, a RAM 7, and a bus 8. The controller 2 is, for example, a CPU, and controls various operations of the information processing apparatus 1 in accordance with programs stored in the storage medium 3. The storage medium is, for example, a hard disk, and stores various data such as the programs necessary for the operations of the information processing apparatus 1 and input images which are images to be processed by the image processor 4. The image processor 4 as the information processing apparatus according to each embodiment of the present invention subjects an input image read from the storage medium 3 and then expanded in the RAM 7 to image processing. The display 5 displays the input image. The operation unit 6 is, for example, a keyboard or a mouse, and is an operational component necessary for a user to perform the various operations of the information processing apparatus 1. The RAM 7 temporarily stores various data generated inside the information processing apparatus 1. The bus 8 is a transfer channel to transfer the various data generated inside the information processing apparatus 1.

Facts

Application

14/980,743Filed 28 Dec 2015 · continuation

Priority

JP 2013-13309925 Jun 2013

Published

US 2016/0148385 A126 May 2016

Granted

5 Sep 2017

Term ends

20 Jun 2034

Estimated · how

20 years from earliest known nonprovisional domestic filing plus recorded patent term adjustment. Maintenance-fee lapses, terminal disclaimers and statutory extensions have not been checked. The continuity chain may be incomplete; this is not an enforceability determination.

Classification11 codes

G06T 7/90
Image analysis: determination of colour characteristics
G06T 5/70
Image enhancement or restoration: denoising; Smoothing
G06T 7/248
Using feature-based methods, e.g. the tracking of corners or segments: involving reference images or patches
G06T 7/74
Using feature-based methods: involving reference images or patches

Timeline

25 Jun 2013Priority claimedJP 2013-133099
28 Dec 2015Application filedUS 14/980,743
28 Dec 2015Assigned to Olympus Corporationassignment of assignor's interest
26 May 2016PublishedUS 2016/0148385 A1
15 Jun 2017Assigned to Olympus Corporationchange of address
5 Sep 2017GrantedUS 9,754,375 B2
12 Oct 2026TodayWithin its term
20 Jun 2034Term endsEstimated

Image processing apparatus, image processing method, and non-transitory storage medium storing image processing program

Life of this patent

Image processing apparatus, image processing method, and non-transitory storage medium storing image processing program

Life of this patent

Life of this patent

Abstract

Drawings

Drawings

Claims13

Description13,070 words

Drawings

Classification11 codes

Life of this patent

Description13,070 words