Image processing apparatus, image processing method, and computer-readable storage medium for calculating a depth value of a subject in an image

An image processing method comprising the steps of:

• extracting, for each pixel of an input image, a high-frequency component as focus information from a brightness signal for a neighboring pixel;
• integrating, for each pixel, the focus information on the neighboring pixel to generate integrated focus information;
• replacing a value of the focus information on each pixel with a value obtained on the basis of a value of the focus information on the neighboring pixel that is greater than the value of the focus information on that pixel to generate dark area focus information on the input image;
• integrating the brightness signal for each pixel with the brightness signal for the neighboring pixel to extract an illumination component;
• calculating a depth value of each pixel through a product-sum operation of the integrated focus information and the dark area focus information using a coefficient based on the illumination component;
• normalizing the depth value of each pixel;
• controlling the depth value normalized in the normalization step in accordance with a tone curve;
• detecting a saturation of each pixel;
• and
• controlling the depth value that has been controlled in accordance with the tone curve in the tone curve control step on the basis of the saturation.

A non-transitory computer-readable storage medium encoded with instructions, which when executed on a processor, perform a method, the method comprising:

• extracting, for each pixel of an input image, a high-frequency component as focus information from a brightness signal for a neighboring pixel;
• integrating, for each pixel, the focus information on the neighboring pixel to generate integrated focus information;
• replacing a value of the focus information on each pixel with a value obtained on the basis of a value of the focus information on the neighboring pixel that is greater than the value of the focus information on that pixel to generate dark area focus information on the input image;
• integrating the brightness signal for each pixel with the brightness signal for the neighboring pixel to extract an illumination component;
• calculating a depth value of each pixel through a product-sum operation of the integrated focus information and the dark area focus information using a coefficient based on the illumination component;
• normalizing the depth value of each pixel;
• controlling the depth value normalized in the normalization step in accordance with a tone curve;
• detecting a saturation of each pixel;
• and
• controlling the depth value that has been controlled in accordance with the tone curve in the tone curve control step on the basis of the saturation.

An image processing apparatus comprising:

• a processor;
• and
• a memory coupled to the processor, the memory including a plurality of sections configured to be executed by the processor, the memory including:
• a focus information extraction section configured to extract, for each pixel of an input image, a high-frequency component as focus information from a brightness signal for a neighboring pixel;
• a region integration section configured to integrate, for each pixel, the focus information on the neighboring pixel to generate integrated focus information;
• a dark area processing section configured to replace a value of the focus information on each pixel with a value obtained on the basis of a value of the focus information on the neighboring pixel that is greater than the value of the focus information on that pixel to generate dark area focus information on the input image;
• an illumination component extraction section configured to integrate the brightness signal for each pixel with the brightness signal for the neighboring pixel to extract an illumination component;
• a bright area/dark area integration section configured to calculate a depth value of each pixel through a product-sum operation of the integrated focus information and the dark area focus information using a coefficient based on the illumination component;
• a normalization section configured to normalize the depth value of each pixel;
• a tone curve control section configured to control the depth value normalized by the normalization section in accordance with a tone curve;
• a saturation detection section configured to detect a saturation of each pixel;
• and
• a saturation control section configured to control the depth value that has been controlled in accordance with the tone curve by the tone curve control section on the basis of the saturation.