Method and Device for Refining a Depth Map

A device comprising:

• a display;
• an image sensor;
• one or more processors;
• a non-transitory memory;
• and
• one or more programs stored in the non-transitory memory, that, when executed by the one or more processors, cause the device to perform a method comprising:
• capturing, using the image sensor, an image of a physical environment;
• obtaining a first depth map including a plurality of depths respectively associated with a plurality of pixels of the image of the physical environment;
• generating a second depth map by aligning one or more portions of the first depth map based on a control signal associated with the image of the physical environment;
• transforming, using the one or more processors, the image of the physical environment based on the second depth map;
• and
• displaying, via the display, the transformed image.

A non-transitory memory storing one or more programs, that, when executed by one or more processors of a device with a display and an image sensor, cause the device to perform a method comprising:

• capturing, using the image sensor, an image of a physical environment;
• obtaining a first depth map including a plurality of depths respectively associated with a plurality of pixels of the image of the physical environment;
• generating a second depth map by aligning one or more portions of the first depth map based on a control signal associated with the image of the physical environment;
• transforming, using the one or more processors, the image of the physical environment based on the second depth map;
• and
• displaying, via the display, the transformed image.