Image processing method, image processing device, and image pickup apparatus

An image processing method comprising:

• acquiring a plurality of viewpoint images;
• generating a plurality of modified viewpoint images by performing processing for expanding difference between the plurality of viewpoint images in every pixel of the plurality of viewpoint images and sharpening parallax, or reducing difference between the plurality of viewpoint images and smoothing the parallax;
• and
• generating an output image from the plurality of modified viewpoint images.

An image pickup apparatus comprising:

• an imaging element in which a plurality of pixels provided with a plurality of sub pixels for receiving light flux passing through different partial pupil areas in a focusing optical system are arrayed;
• a viewpoint image generating unit configured to generate a plurality of viewpoint images in every area of the different partial pupil areas from an input image acquired from the imaging element;
• an captured image generating unit configured to generate an captured image in accordance with a pupil area in which the different partial pupil areas are synthesized from the input image;
• a contrast distribution generating unit configured to generate contrast distribution from the plurality of viewpoint images;
• an output image generating unit configured to generate an output image by performing image processing in accordance with the contrast distribution with respect to an image based on the plurality of viewpoint images.