Input recognition based on distinguishing direct and indirect user interactions
AssigneeApple Inc.
InventorsDavid J. Meyer, Julian K. Shutzberg, David M. Teitelbaum, Daniel J. Brewer, Bharat C. Dandu, Christopher D. McKenzie
Life of this patent
Estimating the term…
Abstract
Various implementations disclosed herein include devices, systems, and methods that interpret user activity as user interactions with user interface (UI) elements positioned within a three-dimensional (3D) space such as an extended reality (XR) environment. Some implementations enable user interactions with virtual elements displayed in 3D environments that utilize alternative input modalities, e.g., XR environments that interpret user activity as either direct interactions or indirect interactions with virtual elements.
Claims20
A method comprising:
- an electronic device having a processor:
- obtaining a position of a virtual object in an extended reality (XR) environment corresponding to a three-dimensional (3D) space;
- obtaining a user hand position in the 3D space based on sensor data, wherein the hand position is associated with a hand gesture;
- determining a distance between the user hand position and the position of the virtual object;
- selecting between a direct interaction mode and an indirect interaction mode based on comparing the distance to a threshold, wherein:
- the direct interaction mode is selected if the distance is below the threshold;
- and
- the indirect interaction mode is selected if the distance is above the threshold, and
- interpreting the hand gesture using an interaction recognition process, wherein a direct interaction recognition process is used when the direct interaction mode is selected and an indirect interaction recognition process is used when the indirect interaction mode is selected.
A system comprising:
- a non-transitory computer-readable storage medium;
- and
- one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the system to perform operations comprising:
- obtaining a position of a virtual object in an extended reality (XR) environment corresponding to a three-dimensional (3D) space;
- obtaining a user hand position in the 3D space based on sensor data, wherein the hand position is associated with a hand gesture;
- determining a distance between the user hand position and the position of the virtual object;
- selecting between a direct interaction mode and an indirect interaction mode based on comparing the distance to a threshold, wherein:
- the direct interaction mode is selected if the distance is below the threshold;
- and
- the indirect interaction mode is selected if the distance is above the threshold, and
- interpreting the hand gesture using an interaction recognition process, wherein a direct interaction recognition process is used when the direct interaction mode is selected and an indirect interaction recognition process is used when the indirect interaction mode is selected.
A non-transitory computer-readable storage medium storing program instructions executable via one or more processors to perform operations comprising:
- obtaining a user hand position in the 3D space based on sensor data, wherein the hand position is associated with a hand gesture;
- determining a distance between the user hand position and the position of the virtual object;
- selecting between a direct interaction mode and an indirect interaction mode based on comparing the distance to a threshold, wherein:
- the direct interaction mode is selected if the distance is below the threshold;
- and
- the indirect interaction mode is selected if the distance is above the threshold, and
- interpreting the hand gesture using an interaction recognition process, wherein a direct interaction recognition process is used when the direct interaction mode is selected and an indirect interaction recognition process is used when the indirect interaction mode is selected.
Dependencies are inferred from the claim text.