Defocus blur removal and depth estimation using dual-pixel image data

A system comprising:

• a processor;
• and
• a non-transitory computer-readable medium having stored thereon instructions that, when executed by the processor, cause the processor to perform operations comprising:
• obtaining dual-pixel image data comprising a first sub-image and a second sub-image;
• determining (i) an in-focus image, (ii) a first blur kernel corresponding to the first sub-image, and (iii) a second blur kernel corresponding to the second sub-image;
• determining a loss value using a loss function comprising one or more of:
• an equivalence loss term configured to determine a difference between (i) a convolution of the first sub-image with the second blur kernel and (ii) a convolution of the second sub-image with the first blur kernel, or a data loss term configured to determine a sum of (i) a difference between the first sub-image and a convolution of the in-focus image with the first blur kernel and (ii) a difference between the second sub-image and a convolution of the in-focus image with the second blur kernel;
• based on the loss value and the loss function, updating one or more of:
• (i) the in-focus image, (ii) the first blur kernel, or (iii) the second blur kernel;
• and
• generating image data based on one or more of:
• (i) the in-focus image as updated, (ii) the first blur kernel as updated, or (iii) the second blur kernel as updated.

A non-transitory computer-readable medium having stored thereon instructions that, when executed by a computing device, cause the computing device to perform operations comprising:

• obtaining dual-pixel image data comprising a first sub-image and a second sub-image;
• determining (i) an in-focus image, (ii) a first blur kernel corresponding to the first sub-image, and (iii) a second blur kernel corresponding to the second sub-image;
• determining a loss value using a loss function comprising one or more of:
• an equivalence loss term configured to determine a difference between (i) a convolution of the first sub-image with the second blur kernel and (ii) a convolution of the second sub-image with the first blur kernel, or a data loss term configured to determine a sum of (i) a difference between the first sub-image and a convolution of the in-focus image with the first blur kernel and (ii) a difference between the second sub-image and a convolution of the in-focus image with the second blur kernel:
• based on the loss value and the loss function, updating one or more of:
• (i) the in-focus image, (ii) the first blur kernel, or (iii) the second blur kernel;
• and
• generating image data based on one or more of:
• (i) the in-focus image as updated, (ii) the first blur kernel as updated, or (iii) the second blur kernel as updated.