Context-aware hazard detection using world-facing cameras in virtual, augmented, and mixed reality (xR) applications

A hardware memory device having program instructions stored thereon that, upon execution by a Head-Mounted Display (HMD) worn by a user during execution of a Virtual, Augmented, or Mixed Reality (xR) application, cause the HMD to:

• receive a frame from a camera mounted on a Head-Mounted Display (HMD) worn by a user;
• associate an object detected in the frame with a Simultaneous Localization and Mapping (SLAM) landmark extracted from an infra-red (IR) image;
• request that the SLAM landmark be maintained within a tracked set of SLAM landmarks;
• in response to a resource utilization reaching a selected value, reduce the tracked set of SLAM landmarks by eliminating one or more SLAM landmarks with low Kalman Gains;
• retain the requested SLAM landmark, wherein the requested SLAM landmark has a lower Kalman Gain than the eliminated one or more SLAM landmarks;
• and
• provide to the user, via the HMD, a safety instruction in response to a distance between the user and the SLAM landmark meeting a threshold.

A method, comprising:

• receiving a frame from a camera mounted on a Head-Mounted Display (HMD) worn by a user;
• associating an object detected in the frame with a Simultaneous Localization and Mapping (SLAM) landmark extracted from an infra-red (IR) image;
• requesting that the SLAM landmark be maintained within a tracked set of SLAM landmarks;
• in response to a resource utilization reaching a selected value, reducing the tracked set of SLAM landmarks by eliminating one or more SLAM landmarks with low Kalman Gains;
• retaining the requested SLAM landmark, wherein the requested SLAM landmark has a lower Kalman Gain than the eliminated one or more SLAM landmarks;
• and
• providing to the user, via the HMD, a safety instruction in response to a distance between the user and the SLAM landmark meeting a threshold.