Realistic occlusion for a head mounted augmented reality display

A method of providing realistic occlusion in a see-through, augmented reality display, comprising:

• capturing image data and depth data for real objects using at least one camera in a user field of view of the see-through, augmented reality display;
• receiving image and depth data, at one or more processors communicatively coupled to the at least one camera, including the user field of view and tracking depth positional data from the user field of view;
• determining, by the one or more processors, a spatial occlusion relationship between an occluding object and an occluded object, and determining a level of detail criteria based on the positional data used in processing a level of occlusion for the occluding object, the occluding object and the occluded object including a real object and a virtual object, respectively, based on the image and depth data;
• and
• rendering the spatial occlusion relationship in the display, by the one or more processors communicatively coupled to the see-through augmented reality display, by modifying display of the virtual object and causing the see-through display to provide varied levels of occlusion as determined by the level of detail criteria when displaying the occluding object.

A see-through, augmented reality display device system for providing realistic occlusion comprising:

• a see-through, augmented reality display having a user field of view and being supported by a support structure of the see-through, augmented reality display device;
• at least one camera supported by the support structure for capturing image data and depth data for real objects in the user field of view of the see-through, augmented reality display;
• one or more processors communicatively coupled to the at least one camera for receiving image and depth data including the user field of view;
• the one or more processors for determining a spatial occlusion relationship between an occluding object and an occluded object, the occluding object and the occluded object including a real object and a virtual object, based on the image and depth data;
• the one or more processors being communicatively coupled to the see-through, augmented reality display and the one or more processors causing the see-through display to represent the spatial occlusion relationship in the display by modifying display of the virtual object;
• earphones attached to the support structure;
• the one or more processors being communicatively coupled to the earphones for controlling output of audio data;
• the one or more processors executing a three dimensional audio engine which modifies audio data for the occluded object in the spatial occlusion relationship based on one or more physical properties associated with the occluding object;
• the one or more processors having access to memory for storing one or more sound occlusion models which represent at least one effect on sound directed at an object based on one or more physical properties of the object;
• and
• the one or more processors executing a three dimensional audio engine which modifies audio data for the occluded object in the partial occlusion based on one or more physical properties associated with the occluding object further comprises:
• the one or more processors executing a three dimensional audio engine modifying the audio data in accordance with the at least one effect on sound represented by the identified sound occlusion model.