Depth prediction from dual pixel images

A computer-implemented method, comprising:

• receiving, at a computing device, a dual pixel image of at least a foreground object, the dual pixel image comprising a plurality of dual pixels, wherein a dual pixel of the plurality of dual pixels comprises a left-side pixel and a right-side pixel that both represent light incident on a single dual pixel element used to capture the dual pixel image;
• providing the dual pixel image to a trained machine learning system to determine a depth map for the plurality of dual pixels of the dual pixel image, wherein the trained machine learning system comprises a neural network trained using training images and a loss function that is invariant to depth ambiguities caused by lens characteristics of a lens used to capture the training images;
• and
• determining an output image based on the dual pixel image and the depth map using the computing device.

A computing device, comprising:

• one or more processors;
• and
• data storage, wherein the data storage has stored thereon computer-executable instructions that, when executed by the one or more processors, cause the computing device to carry out functions comprising:
• receiving a dual pixel image of at least a foreground object, the dual pixel image comprising a plurality of dual pixels, wherein a dual pixel of the plurality of dual pixels comprises a left-side pixel and a right-side pixel that both represent light incident on a single dual pixel element used to capture the dual pixel image;
• providing the dual pixel image to a trained machine learning system to determine a depth map for the plurality of dual pixels of the dual pixel image, wherein the trained machine learning system comprises a neural network trained using training images and a loss function that is invariant to depth ambiguities caused by lens characteristics of a lens used to capture the training images;
• and
• determining an output image based on the dual pixel image and the depth map.