Systems, Methods, and Media for Generating Visualization of Physical Environment in Artificial Reality

One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

• receive an image and depth measurements of an environment surrounding a user;
• generate a 3D model representing the environment based on the depth measurements;
• identify, in the image, first pixels depicting a physical object in the environment and second pixels corresponding to a padded boundary around the first pixels;
• associate the first pixels with (1) a first portion of the 3D model representing the physical object and (2) a first representative depth value computed based on the depth measurements;
• associate the second pixels with (1) a second portion of the 3D model representing a region around the physical object and (2) a second representative depth value farther than the first representative depth value;
• and
• render an output image depicting a virtual object and the physical object, wherein occlusions between the virtual object and the physical object are determined using the first representative depth value and the second representative depth value.

A system comprising:

• one or more processors;
• and
• one or more computer-readable non-transitory storage media coupled to one or more of the processors and comprising instructions operable when executed by one or more of the processors to cause the system to:
• receive an image and depth measurements of an environment surrounding a user;
• generate a 3D model representing the environment based on the depth measurements;
• identify, in the image, first pixels depicting a physical object in the environment and second pixels corresponding to a padded boundary around the first pixels;
• associate the first pixels with (1) a first portion of the 3D model representing the physical object and (2) a first representative depth value computed based on the depth measurements;
• associate the second pixels with (1) a second portion of the 3D model representing a region around the physical object and (2) a second representative depth value farther than the first representative depth value;
• and
• render an output image depicting a virtual object and the physical object, wherein occlusions between the virtual object and the physical object are determined using the first representative depth value and the second representative depth value.