Occlusion of virtual objects in augmented reality by physical objects

One or more computer-readable non-transitory storage media embodying software that is operable when executed to perform operations comprising:

• accessing an image comprising a hand of a user of a head-mounted display;
• generating, from at least the image, a planar virtual object representation of the hand of the user, the planar virtual object representation of the hand being defined in a virtual environment that comprises at least one other virtual object;
• determining that a depth of a portion of the planar virtual object representation of the hand to a viewpoint of the user is less than a depth of the other virtual object to the viewpoint of the user;
• rendering, based on the planar virtual object representation of the hand and the other virtual object in the virtual environment, an image of the virtual environment from the viewpoint of the user, the image comprising a set of pixels that corresponds to the portion of the planar virtual object representation of the hand with the depth that is less than the depth of the other virtual object to the viewpoint of the user;
• and
• providing, to a set of light emitters of the head-mounted display, instructions to display the image of the virtual environment, wherein the set of pixels in the image that corresponds to a portion of the planar virtual object representation of the hand cause the light emitters at one or more positions to be unilluminated.

A system comprising:

• one or more processors;
• and
• one or more computer-readable non-transitory storage media coupled to one or more of the processors and comprising instructions operable when executed by one or more of the processors to cause the system to perform operations comprising:
• accessing an image comprising a hand of a user of a head-mounted display;
• generating, from at least the image, a planar virtual object representation of the hand of the user, the planar virtual object representation of the hand being defined in a virtual environment that comprises at least one other virtual object;
• determining that a depth of a portion of the planar virtual object representation of the hand to a viewpoint of the user is less than a depth of the other virtual object to the viewpoint of the user;
• rendering, based on the planar virtual object representation of the hand and the other virtual object in the virtual environment, an image of the virtual environment from the viewpoint of the user, the image comprising a set of pixels that corresponds to the portion of the planar virtual object representation of the hand with the depth that is less than the depth of the other virtual object to the viewpoint of the user;
• and
• providing, to a set of light emitters of the head-mounted display, instructions to display the image of the virtual environment, wherein the set of pixels in the image that corresponds to a portion of the planar virtual object representation of the hand cause light emitters at one or more positions to be unilluminated.