Image processing apparatus and image processing method

An image processing apparatus for performing image processing using a plurality of captured images acquired by capturing a same subject in a plurality of imaging states, the image processing apparatus comprising:

• a non-transitory memory device storing a program;
• and
• a hardware processor configured to execute the program and cause the image processing apparatus to operate as:
• a target motion amount estimation unit configured to estimate a target motion amount that represents an amount of shift in the position of the subject between a first image and a second image among the captured images, the first image being captured in a first imaging state, and the second image being captured in a second imaging state different from the first imaging state;
• a corrected image generation unit configured to generate a corrected image by performing motion compensation on the second image based on the target motion amount;
• and
• an image processing unit configured to perform image processing using the first image and the corrected image,
• wherein the image processing apparatus receives the first image, the second image, and a third image, the second image being captured during a time interval between the first image and the third image, and the third image being captured in the first focus state at a time different from a time when the first image is captured, and
• the target motion amount estimation unit includes:
• a first motion amount estimation unit configured to estimate a first motion amount that represents an amount of shift in a position of the subject between the first image and the third image;
• and
• a target motion amount determination unit configured to estimate the target motion amount using the first motion amount, and
• wherein the target motion amount estimation unit further includes a second motion amount estimation unit configured to estimate a second motion amount that represents an amount of position shift between the first image and the second image,
• the target motion amount determination unit is configured to estimate the target motion amount using the first motion amount and the second motion amount,
• the target motion amount determination unit is configured to determine accuracy of the second motion amount based on a difference in pixel value between a calculation target pixel that is included in the second image and for which the target motion amount is to be calculated among pixels that constitute the second image and a pixel that is included in the first image and corresponds to the calculation target pixel, and when it is determined that the accuracy of the second motion amount is higher than a threshold value, estimate the second motion amount as the target motion amount, and when it is determined that the accuracy of the second motion amount is lower than the threshold value, estimate the target motion amount using the first motion amount.

An image processing method of an image processing apparatus for measuring subject distance using a plurality of captured images acquired by capturing a same subject in a plurality of focus states, the image processing method comprising:

• estimating a target motion amount that represents an amount of shift in the position of the subject between a first image and a second image among the captured images, the first image being captured in a first focus state, and the second image being captured in a second focus state different from the first focus state;
• generating a corrected image by performing motion compensation on the second image based on the target motion amount;
• and
• measuring a subject distance for the first image based on a correlation value in out-of-focus amount between the first image and the corrected image,
• wherein the image processing method receives the first image, the second image, and a third image, the second image being captured during a time interval between the first image and the third image, and the third image being captured in the first focus state at a time different from a time when the first image is captured, and
• estimating the target motion amount includes:
• estimating a first motion amount that represents an amount of shift in a position of the subject between the first image and the third image;
• and
• estimating the target motion amount using the first motion amount, and
• wherein the image processing apparatus includes a non-transitory memory device storing a program, and a hardware processor configured to execute the program so as to perform the image processing method, and
• wherein the estimating of the target motion amount includes estimating a second motion amount that represents an amount of position shift between the first image and the second image,
• the determining of the target motion amount estimates the target motion amount using the first motion amount and the second motion amount, and
• the determining of the target motion amount determines accuracy of the second motion amount based on a difference in pixel value between a calculation target pixel that is included in the second image and for which the target motion amount is to be calculated among pixels that constitute the second image and a pixel that is included in the first image and corresponds to the calculation target pixel, and when it is determined that the accuracy of the second motion amount is higher than a threshold value, estimates the second motion amount as the target motion amount, and when it is determined that the accuracy of the second motion amount is lower than the threshold value, estimates the target motion amount using the first motion amount.

An image processing method for performing image processing using a plurality of captured images acquired by capturing the same subject in a plurality of imaging states, the image processing method comprising:

• estimating a target motion amount that represents an amount of shift in the position of the subject between a first image and a second image among the captured images, the first image being captured in a first imaging state, and the second image being captured in a second imaging state different from the first imaging state;
• generating a corrected image by performing motion compensation on the second image based on the target motion amount;
• and
• performing image processing using the first image and the corrected image,
• wherein the image processing method receives the first image, the second image, and a third image, the second image being captured during a time interval between the first image and the third image, and the third image being captured in the first focus state at a time different from a time when the first image is captured, and
• estimating the target motion amount includes:
• estimating a first motion amount that represents an amount of shift in a position of the subject between the first image and the third image;
• and
• estimating the target motion amount using the first motion amount, and
• wherein the image processing apparatus includes a non-transitory memory device storing a program, and a hardware processor configured to execute the program so as to perform the image processing method, and
• wherein the estimating of the target motion amount includes estimating a second motion amount that represents an amount of position shift between the first image and the second image, and
• the determining of the target motion amount estimates the target motion amount using the first motion amount and the second motion amount, and
• the determining of the target motion amount determines accuracy of the second motion amount based on a difference in pixel value between a calculation target pixel that is included in the second image and for which the target motion amount is to be calculated among pixels that constitute the second image and a pixel that is included in the first image and corresponds to the calculation target pixel, and when it is determined that the accuracy of the second motion amount is higher than a threshold value, estimates the second motion amount as the target motion amount, and when it is determined that the accuracy of the second motion amount is lower than the threshold value, estimates the target motion amount using the first motion amount.