Image processing apparatus, image capturing apparatus, image processing method, and storage medium

An image processing apparatus comprising:

• one or more processors;
• and
• a memory storing instructions which, when executed by the one or more processors, cause the image processing apparatus to function as:
• an obtaining unit configured to obtain a distance map and a shot image, the distance map representing a distribution of information related to a distance in a depth direction of a shooting range using a distribution of distance values, the shot image corresponding to the shooting range,
• a generation unit configured to generate an object region map which indicates an object region having a distance value in a predetermined distance value range in the distance map, wherein in the object region map, the object region has a pixel value included in a first pixel value range and a region other than the object region has a pixel value included in a second pixel value range that does not overlap the first pixel value range,
• a first filter unit configured to apply a first filter to the object region map,
• a second filter unit configured to apply a second filter to the object region map to which the first filter has been applied, and
• a third filter unit configured to apply a third filter to the object region map generated by the generation unit before the first filter is applied thereto,
• wherein a number of bits for a pixel value in the object region map is smaller than a number of bits for a distance value in the distance map,
• the first filter is a filter that reduces the object region in the object region map,
• the second filter is a filter that changes a pixel value of a target pixel in the object region map so that a similarity between the target pixel and a plurality of surrounding pixels in the object region map becomes close to a similarity between the target pixel and the plurality of surrounding pixels in the shot image, and
• the third filter is a filter that changes a pixel value of a target pixel that deviates from a tendency of a plurality of surrounding pixel values in the object region map so as to reduce an extent of the deviation from the tendency.

An image processing method executed by an image processing apparatus, comprising:

• obtaining a distance map and a shot image, the distance map representing a distribution of information related to a distance in a depth direction of a shooting range using a distribution of distance values, the shot image corresponding to the shooting range,
• applying a first filter to the distance map,
• applying a second filter to the distance map to which the first filter has been applied, and
• applying a third filter unit configured to apply a third filter to the distance map obtained by the obtaining unit before the first filter is applied thereto,
• wherein the first filter is a filter that reduces a distribution of distance values in a predetermined distance value range in the distance map,
• the second filter is a filter that changes a distance value of a target distance in the distance map using weighted averaging that gives more weight to a distance value that is closer in position to the target distance, and
• the third filter is a filter that changes a pixel value of a target pixel that deviates from a tendency of a plurality of surrounding pixel values in the object region map so as to reduce an extent of the deviation from the tendency.

An image processing method executed by an image processing apparatus, comprising:

• obtaining a distance map and a shot image, the distance map representing a distribution of information related to a distance in a depth direction of a shooting range using a distribution of distance values, the shot image corresponding to the shooting range,
• generating an object region map which indicates an object region having a distance value in a predetermined distance value range in the distance map, wherein in the object region map, the object region has a pixel value included in a first pixel value range and a region other than the object region has a pixel value included in a second pixel value range that does not overlap the first pixel value range,
• applying a first filter to the object region map,
• applying a second filter to the object region map to which the first filter has been applied,
• applying a third filter unit configured to apply a third filter to the distance map obtained by the obtaining unit before the first filter is applied thereto,
• wherein a number of bits for a pixel value in the object region map is smaller than a number of bits for a distance value in the distance map,
• the first filter is a filter that reduces the object region in the object region map,
• the second filter is a filter that changes a pixel value of a target pixel in the object region map so that a similarity between the target pixel and a plurality of surrounding pixels in the object region map becomes close to a similarity between the target pixel and the plurality of surrounding pixels in the shot image, and
• the third filter is a filter that changes a distance value of a target position that deviates from a tendency of a plurality of surrounding distance values in the distance map so as to reduce an extent of the deviation from the tendency.

A non-transitory computer-readable storage medium which stores a program for causing a computer to execute an image processing method comprising:

• obtaining a distance map and a shot image, the distance map representing a distribution of information related to a distance in a depth direction of a shooting range using a distribution of distance values, the shot image corresponding to the shooting range,
• applying a first filter to the distance map,
• applying a second filter to the distance map to which the first filter has been applied, and
• applying a third filter unit configured to apply a third filter to the distance map obtained by the obtaining unit before the first filter is applied thereto,
• wherein the first filter is a filter that reduces a distribution of distance values in a predetermined distance value range in the distance map,
• the second filter is a filter that changes a distance value of a target distance in the distance map using weighted averaging that gives more weight to a distance value that is closer in position to the target distance, and
• the third filter is a filter that changes a distance value of a target position that deviates from a tendency of a plurality of surrounding distance values in the distance map so as to reduce an extent of the deviation from the tendency.

A non-transitory computer-readable storage medium which stores a program for causing a computer to execute an image processing method comprising:

• obtaining a distance map and a shot image, the distance map representing a distribution of information related to a distance in a depth direction of a shooting range using a distribution of distance values, the shot image corresponding to the shooting range,
• generating an object region map which indicates an object region having a distance value in a predetermined distance value range in the distance map, wherein in the object region map, the object region has a pixel value included in a first pixel value range and a region other than the object region has a pixel value included in a second pixel value range that does not overlap the first pixel value range,
• applying a first filter to the object region map,
• applying a second filter to the object region map to which the first filter has been applied, and
• applying a third filter unit configured to apply a third filter to the distance map obtained by the obtaining unit before the first filter is applied thereto,
• wherein a number of bits for a pixel value in the object region map is smaller than a number of bits for a distance value in the distance map,
• the first filter is a filter that reduces the object region in the object region map,
• the second filter is a filter that changes a pixel value of a target pixel in the object region map so that a similarity between the target pixel and a plurality of surrounding pixels in the object region map becomes close to a similarity between the target pixel and the plurality of surrounding pixels in the shot image, and
• the third filter is a filter that changes a distance value of a target position that deviates from a tendency of a plurality of surrounding distance values in the distance map so as to reduce an extent of the deviation from the tendency.