Systems, methods, and media for generating visualization of physical environment in artificial reality

One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

• compute, based on at least one image of a physical environment surrounding a user, depth measurements of at least one physical object;
• generate a first model of the at least one physical object using the depth measurements;
• identify a plurality of first pixels in the image that depict the physical object;
• associate the plurality of first pixels with a first representative depth value computed using the first model;
• determine, for a pixel of an output image, that a portion of the first model and a portion of a second model of a virtual object are visible from a viewpoint;
• determine that the portion of the first model is associated with the plurality of first pixels;
• and
• render the output image depicting the physical object and the virtual object from the perspective of the user, wherein occlusion at the pixel is determined based on a comparison between the first representative depth value and a depth value associated with the portion of the second model.

A system comprising:

• one or more processors;
• and
• one or more computer-readable non-transitory storage media coupled to one or more of the processors and comprising instructions operable when executed by one or more of the processors to cause the system to:
• compute, based on at least one image of a physical environment surrounding a user, depth measurements of at least one physical object;
• generate a first model of the at least one physical object using the depth measurements;
• identify a plurality of first pixels in the image that depict the physical object;
• associate the plurality of first pixels with a first representative depth value computed using the first model;
• determine, for a pixel of an output image, that a portion of the first model and a portion of a second model of a virtual object are visible from a viewpoint;
• determine that the portion of the first model is associated with the plurality of first pixels;
• and
• render the output image depicting the physical object and the virtual object from the perspective of the user, wherein occlusion at the pixel is determined based on a comparison between the first representative depth value and a depth value associated with the portion of the second model.