Holographic palm raycasting for targeting virtual objects

A method, comprising:

• receiving information from a head-mounted depth camera about an environment;
• presenting one or more virtual objects on a head-mounted display so that the virtual objects appear to be positioned within the environment;
• determining a position of the head-mounted display within the environment;
• assigning a position within the environment of a joint of a user's arm that is located outside of a field-of-view of the depth camera, the assigned position based at least on the determined position of the head-mounted display and a pre-determined spatial relationship between the joint of the user's arm and the head-mounted display;
• determining a position of the user's hand based at least on pixels of a depth map corresponding to the user as determined from the information received from the depth camera;
• casting a ray from a portion of the user's hand based on the position of the joint of the user's arm and the position of the user's hand;
• and
• responsive to the ray intersecting with one or more control points of a virtual object, indicating to the user that the virtual object is being targeted.

A system for a head-mounted display, comprising:

• a frame;
• a display device;
• an outward-facing depth camera positioned on the frame;
• and
• a storage machine comprising instructions executable by a logic machine to:
• receive information from the depth camera about an environment;
• determine a position of the head-mounted display within the environment;
• present one or more virtual objects on the display device so that the one or more virtual objects appear to be positioned within a threshold distance of the head-mounted display;
• select a first virtual object within the threshold distance of the head-mounted display responsive to recognizing a natural selection gesture by a user's hand based on information received from the depth camera;
• recognize a first manipulation gesture selected from a first set of manipulation gestures from fingers of the user's hand based on information received from the depth camera;
• manipulate the first virtual object responsive to recognizing the first manipulation gesture;
• present one or more virtual objects on the display device so that the one or more virtual objects appear to be positioned at a distance greater than the threshold distance from the head-mounted display;
• assign a position within the environment of a joint of a user's arm that is located outside of a field-of-view of the depth camera, the assigned position based at least on the determined position of the head-mounted display and a pre-determined spatial relationship between the joint of the user's arm and the head-mounted display;
• determine a position of the user's hand based at least on pixels of a depth map corresponding to the user as determined from the information received from the depth camera;
• cast a ray from a portion of the user's hand based on the position of the joint of the user's arm and the position of the user's hand;
• responsive to the ray intersecting with one or more control points of a second virtual object that is greater than the threshold distance from the head-mounted display, indicate to the user that the second virtual object is being targeted;
• recognize a selection gesture from the user's hand based on information received from the depth camera;
• select the second virtual object responsive to recognizing the selection gesture;
• recognize a second manipulation gesture selected from the first set of manipulation gestures from fingers of the user's hand based on information received from the depth camera;
• and
• manipulate the second virtual object responsive to recognizing the second manipulation gesture.