Depth from motion for augmented reality for handheld user devices

A handheld user device, comprising:

• a monocular camera to capture a feed of images of a local scene;
• a display panel;
• a memory to store a software application;
• and
• a processor coupled to the memory and to the monocular camera, wherein the processor is to execute instructions of the software application to:
• select, from the feed, a keyframe;
• perform, for a first image from the feed of images, stereo matching using the first image, the keyframe, and a relative pose based on a pose associated with the first image and a pose associated with the keyframe to generate a sparse disparity map representing disparities between the first image and the keyframe;
• determine a dense depth map from the disparity map using a bilateral solver algorithm;
• process a viewfinder image generated from a second image of the feed with occlusion rendering based on the depth map to incorporate one or more virtual objects into the viewfinder image to generate an AR viewfinder image;
• and
• provide the AR viewfinder image for display at the display panel.

A non-transitory computer-readable storage medium storing a set of executable instructions, the set of executable instructions configured to manipulate a processor of a handheld user device to:

• select a keyframe from a feed of images captured by a monocular camera;
• perform, for a first image from the feed of images, stereo matching using the first image, the keyframe, and a relative pose based on a pose associated with the first image and a pose associated with the keyframe to generate a sparse disparity map representing disparities between the first image and the keyframe;
• determine a dense depth map from the disparity map using a bilateral solver algorithm;
• process a viewfinder image generated from a second image of the feed with occlusion rendering based on the depth map to incorporate one or more virtual objects into the viewfinder image to generate an AR viewfinder image;
• and
• provide the AR viewfinder image for display at a display panel of the handheld user device.