System and method for parallax correction for video see-through augmented reality

A video see-through (VST) augmented reality (AR) device comprising:

• at least one display panel;
• first and second see-through cameras;
• and
• at least one processing device configured to:
• obtain a stereo image pair including a first image and a second image captured using the first and second see-through cameras;
• generate a first feature map of the first image and a second feature map of the second image, the first feature map including extracted positions associated with pixels in the first image, the second feature map including extracted positions associated with pixels in the second image;
• generate a disparity map between the first and second images based on a dense depth map;
• generate a verified depth map based on a pixelwise comparison of predicted positions and the extracted positions associated with at least some of the pixels in at least one of the first and second images, the predicted positions determined based on the disparity map, and wherein, for each pixel in the first image that represents a point in space, either a depth value or a corrected depth value is stored in the verified depth map based on a threshold;
• and
• generate a first virtual view and a second virtual view to present on a display panel of the VST AR device based on the verified depth map, wherein to generate the first virtual view and the second virtual view based on the verified depth map, the at least one processing device is configured to warp the verified depth map to generate a first warped depth map for a first virtual camera and a second warped depth map for a second virtual camera.

A non-transitory machine readable medium containing instructions that when executed cause at least one processor to:

• obtain a stereo image pair including a first image and a second image captured using first and second see-through cameras associated with a video see-through (VST) augmented reality (AR) device;
• generate a first feature map of the first image and a second feature map of the second image, the first feature map including extracted positions associated with pixels in the first image, the second feature map including extracted positions associated with pixels in the second image;
• generate a disparity map between the first and second images based on a dense depth map;
• generate a verified depth map based on a pixelwise comparison of predicted positions and the extracted positions associated with at least some of the pixels in at least one of the first and second images, the predicted positions determined based on the disparity map, and wherein, for each pixel in the first image that represents a point in space, either a depth value or a corrected depth value is stored in the verified depth map based on a threshold;
• and
• generate a first virtual view and a second virtual view to present on a display panel of the VST AR device based on the verified depth map, wherein the instructions that cause the at least one processor to generate the first virtual view and the second virtual view based on the verified depth map comprise instructions to warp the verified depth map to generate a first warped depth map for a first virtual camera and a second warped depth map for a second virtual camera.