Generating and modifying an artificial reality environment using occlusion surfaces at predetermined distances

One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

• generate a depth map of a real environment as seen from a viewpoint of a user, the depth map comprising pixels having corresponding depth values of one or more physical objects in the real environment;
• generate, based on the depth map, a two-dimensional occlusion surface representing at least a visible portion of the one or more physical objects that are located within a predetermined depth range defined relative to the viewpoint;
• pose the two-dimensional occlusion surface in a three-dimensional coordinate system such that the two-dimensional occlusion surface is located at a predetermined distance from the viewpoint;
• determine a visibility of a first virtual object relative to the one or more physical objects by comparing a model of the virtual object with the two-dimensional occlusion surface;
• identify, based on the depth map, a first physical object of the one or more physical objects that is located at a depth that is less than the depth range defined relative to the viewpoint;
• generate a pre-occluded two-dimensional surface representing a second virtual object occluded by the first physical object, wherein the pre-occluded two-dimensional surface is based on a visibility of the first physical object from the viewpoint of a user;
• pose the pre-occluded two-dimensional surface in the three-dimensional coordinate system;
• and
• generate an output image based on the determined visibility of the first virtual object and the posed pre-occluded two-dimensional surface representing the second virtual object occluded by the first physical object.

A system comprising:

• one or more processors;
• and
• one or more computer-readable non-transitory storage media coupled to one or more of the processors and comprising instructions operable when executed by one or more of the processors to cause the system to:
• generate a depth map of a real environment as seen from a viewpoint of a user, the depth map comprising pixels having corresponding depth values of one or more physical objects in the real environment;
• generate, based on the depth map, a two-dimensional occlusion surface representing at least a visible portion of the one or more physical objects that are located within a predetermined depth range defined relative to the viewpoint;
• pose the two-dimensional occlusion surface in a three-dimensional coordinate system such that the two-dimensional occlusion surface is located at a predetermined distance from the viewpoint;
• determine a visibility of a first virtual object relative to the one or more physical objects by comparing a model of the virtual object with the two-dimensional occlusion surface;
• identify, based on the depth map, a first physical object of the one or more physical objects that is located at a depth that is less than the depth range defined relative to the viewpoint;
• generate a pre-occluded two-dimensional surface representing a second virtual object occluded by the first physical object, wherein the pre-occluded two-dimensional surface is based on a visibility of the first physical object from the viewpoint of a user;
• pose the pre-occluded two-dimensional surface in the three-dimensional coordinate system;
• and
• generate an output image based on the determined visibility of the first virtual object and the posed pre-occluded two-dimensional surface representing the second virtual object occluded by the first physical object.