Input Recognition in 3D Environments

A system comprising:

• a non-transitory computer-readable storage medium;
• and
• one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the system to perform operations comprising:
• obtaining hand data associated with a position of a hand in a three-dimensional (3D) space, the hand data obtained based on first sensor data;
• obtaining gaze data associated with a direction of a gaze in the 3D space, the gaze data obtained based on second sensor data;
• selecting a direct interaction mode;
• in accordance with selecting the direct interaction mode, identifying a first interaction with a first virtual object based on identifying that the position of the hand intersects a first area associated with a position of the first virtual object in an extended reality (XR) environment corresponding to the 3D space;
• selecting an indirect interaction mode;
• and
• in accordance with selecting the indirect interaction mode, identifying a second interaction with a second virtual object based on identifying that the gaze direction intersects a second area associated with a position of the second virtual object in the XR environment.

A non-transitory computer-readable storage medium storing program instructions executable via one or more processors to perform operations comprising:

• obtaining hand data associated with a position of a hand in a three-dimensional (3D) space, the hand data obtained based on first sensor data;
• obtaining gaze data associated with a direction of a gaze in the 3D space, the gaze data obtained based on second sensor data;
• selecting a direct interaction mode;
• in accordance with selecting the direct interaction mode, identifying a first interaction with a first virtual object based on identifying that the position of the hand intersects a first area associated with a position of the first virtual object in an extended reality (XR) environment corresponding to the 3D space;
• selecting an indirect interaction mode;
• and
• in accordance with selecting the indirect interaction mode, identifying a second interaction with a second virtual object based on identifying that the gaze direction intersects a second area associated with a position of the second virtual object in the XR environment.