Image processing apparatus, image processing method, image capturing apparatus, and storage medium for panoramic image composition

An image pickup apparatus, comprising:

• an image sensor configured to pick up a plurality of images;
• at least one memory configured to store instructions;
• and
• at least one processor in communication with the at least one memory, configured to execute the instructions to:
• execute first calculation in which a conversion coefficient for executing positional adjustment of images captured at a same exposure is calculated with respect to a plurality of images captured at different exposures;
• execute second calculation different from the first calculation, in which a conversion coefficient for executing positional adjustment of images captured at different exposures is calculated, with respect to the plurality of images;
• execute positional adjustment with respect to the plurality of images;
• and
• execute composition with respect to the images on which the positional adjustment is executed to composite a panoramic image having a dynamic range and a viewing angle wider than a dynamic range and a viewing angle of the image,
• wherein, the at least one processor further executes the instructions to, in the second calculation, when calculating a conversion coefficient with respect to an image in a part of images from the plurality of images, newly generate an image used for detection having an exposure that is the same as an exposure of an image that is to be positionally adjusted to the image in the part of images by using an exposure of the image and calculate a conversion coefficient by using the image used for detection, and to calculate a conversion coefficient for executing positional adjustment based on the conversion coefficient calculated in the first calculation with respect to images other than the part of images from the plurality of images.

A control method to control an image processing apparatus, the method comprising:

• executing first calculation in which a conversion coefficient for executing positional adjustment of images captured at a same exposure is calculated with respect to a plurality of images captured at different exposures;
• executing second calculation different from the first calculation, in which a conversion coefficient for executing positional adjustment of images captured at different exposures is calculated with respect to the plurality of images;
• executing positional adjustment with respect to the plurality of images;
• and
• executing composition with respect to the images on which the positional adjustment is executed to composite a panoramic image having a dynamic range and a viewing angle wider than a dynamic range and a viewing angle of the image,
• wherein, in the second calculation, when calculating a conversion coefficient with respect to an image in a part of images from the plurality of images, an image used for detection having an exposure that is the same as an exposure of an image that is to be positionally adjusted to the image is newly generated in the part of images by using an exposure of the image and calculate a conversion coefficient by using the image used for detection, and a conversion coefficient is calculated for executing positional adjustment based on the conversion coefficient calculated in the first calculation with respect to images other than the part of images from the plurality of images.

A computer-readable storage medium storing instructions that cause a computer to execute a method for controlling an image processing apparatus, the method comprising:

• executing first calculation in which a conversion coefficient for executing positional adjustment of images captured at a same exposure is calculated with respect to a plurality of images captured at different exposures;
• executing second calculation different from the first calculation, in which a conversion coefficient for executing positional adjustment of images captured at different exposures is calculated with respect to the plurality of images;
• executing positional adjustment with respect to the plurality of images;
• and
• executing composition with respect to the images on which the positional adjustment is executed to composite a panoramic image having a dynamic range and a viewing angle wider than a dynamic range and a viewing angle of the image,
• wherein, in the second calculation, when calculating a conversion coefficient with respect to an image in a part of images from the plurality of images, an image used for detection having an exposure that is the same as an exposure of an image that is to be positionally adjusted to the image is newly generated in the part of images by using an exposure of the image and calculate a conversion coefficient by using the image used for detection, and a conversion coefficient is calculated for executing positional adjustment based on the conversion coefficient calculated in the first calculation with respect to images other than the part of images from the plurality of images.