Depth-varying reprojection passthrough in video see-through (VST) extended reality (XR)

An extended reality (XR) device comprising:

• at least one display;
• imaging sensors configured to capture images of a scene, the scene including one or more first objects and one or more second objects;
• and
• at least one processing device configured to:
• obtain (i) the images of the scene and (ii) depth data associated with the images;
• obtain volume-based three-dimensional (3D) models of the first and second objects included in the scene, the volume-based 3D models including:
• (i) one or more 3D models of the one or more first objects and (ii) one or more 3D models of the one or more second objects;
• for the one or more first objects, perform depth-based reprojection of the one or more 3D models of the one or more first objects to a left virtual view and a right virtual view based on one or more depths of the one or more first objects;
• for the one or more second objects, perform constant-depth reprojection of the one or more 3D models of the one or more second objects to the left virtual view and the right virtual view based on a specified depth;
• and
• render the left virtual view and the right virtual view for presentation by the at least one display.

A non-transitory machine readable medium containing instructions that when executed cause at least one processor to:

• obtain (i) images of a scene captured using a stereo pair of imaging sensors of an extended reality (XR) device and (ii) depth data associated with the images, the scene including one or more first objects and one or more second objects;
• obtain volume-based three-dimensional (3D) models of the first and second objects included in the scene, the volume-based 3D models including:
• (i) one or more 3D models of the one or more first objects and (ii) one or more 3D models of the one or more second objects;
• for the one or more first objects, perform depth-based reprojection of the one or more 3D models of the one or more first objects to a left virtual view and a right virtual view based on one or more depths of the one or more first objects;
• for the one or more second objects, perform constant-depth reprojection of the one or more 3D models of the one or more second objects to the left virtual view and the right virtual view based on a specified depth;
• and
• render the left virtual view and the right virtual view for presentation by the XR device.