Image processing apparatus, image processing method, and program thereof

An image processing apparatus which extracts feature points for image alignment by which a vertical disparity between a first image and a second image is reduced, the first image and the second image being images of a same object captured from different viewpoints, the image processing apparatus comprising:

• a non-transitory memory storing a program;
• and
• a hardware processor configured to execute the program and to cause the image processing apparatus to operate as the following units stored in the non-transitory memory:
• a division unit configured to divide each of the first image and the second image into a plurality of sub-regions;
• and
• an extraction unit configured to (i) perform feature point extraction on each of the sub-regions included in the first image and on each of the sub-regions included in the second image, and (ii) output coordinates of a feature point extracted by the feature point extraction, for every one of the sub-regions included in the first image and the second image,
• wherein the extraction unit is configured to perform the feature point extraction in such a manner that a value indicating a degree of variation among the sub-regions in a total number of feature points extracted by the feature point extraction is less than or equal to a predetermined value,
• the extraction unit is configured to perform the feature point extraction in such a manner so as to extract, from each of the sub-regions, as many feature points as necessary for making a difference between a predetermined number N and the total number of feature points extracted from the sub-region less than or equal to a predetermined value,
• the extraction unit includes:
• a feature amount calculation unit configured to calculate, for each of the sub-regions, feature amounts each corresponding to one of a plurality of pixels included in the sub-region;
• a feature point extraction unit configured to extract, as a feature point, a pixel among the pixels which has a feature amount greater than or equal to a predetermined feature point threshold;
• an adjustment unit configured to adjust the feature point threshold for each of the sub-regions so that, for each sub-region, a difference between the predetermined number N and a total number of feature points extracted by the feature point extraction unit is less than or equal to the predetermined value;
• and
• a fake feature point generation unit configured to generate a fake feature point in a first sub-region included in the sub-regions, based on feature points extracted by the feature point extraction unit,
• the fake feature point generation unit is configured to generate, in the first sub-region, as many fake feature points as necessary for making a difference between the predetermined number N and a sum of a total number of feature points extracted from the first sub-region and a total number of fake feature points in the first sub-region less than or equal to the predetermined value, and
• the fake feature point generation unit includes:
• a first fake point generation unit configured to generate a first fake point based on a first feature point and a second feature point which are included in the first image, the first fake point being a fake feature point in the first image;
• a reference information obtaining unit configured to obtain reference information including a distance in the first image between the first fake point and the first feature point and a distance in the first image between the first fake point and the second feature point;
• a corresponding point obtaining unit configured to obtain (i) a third feature point which is included in the second image and corresponds to the first feature point and (ii) a fourth feature point which is included in the second image and corresponds to the second feature point;
• and
• a second fake point generation unit configured to generate, in the second image, a second fake point as a fake feature point corresponding to the first fake point, by referring to the reference information.