XR virtual trackpad
AssigneeNtt Docomo, Inc.
InventorsYoung-Cheol Yu, Yuya Tanaka, Yoji Osako
Life of this patent
Estimating the term…
Abstract
A virtual trackpad and method for using the same are disclosed. In some embodiments, the method comprises: projecting a virtual trackpad on one hand of a user using hand tracking position data from an extend reality (XR) device; detecting, via hand tracking, movements of one or more fingers as the one or more fingers are being moved across the virtual trackpad projected on the one hand; and controlling a user interface (UI) being displayed on a display by the XR device based on the movements of the one or more fingers.
Claims20
A method comprising:
- determining the user is looking at the one hand using image recognition and hand tracking;
- generating a digital model of the one hand by converting captured data on position, orientation, and velocity of the one hand in response to determining the user is looking at the one hand;
- identifying a position on the one hand from the digital model;
- projecting a virtual trackpad at the position on the one hand of a user using hand tracking position data from an extend reality (XR) device in response to determining the user is looking at the one hand;
- detecting, via hand tracking, movements of one or more fingers as the one or more fingers are being moved across the virtual trackpad projected on the one hand;
- and
- controlling a user interface (UI) being displayed on a display by the XR device based on the movements of the one or more fingers.
A computing system for controlling a user interface, the computing system comprising:
- one or more processors;
- and
- one or more memories storing instructions that, when executed by the one or more processors, cause the computing system to perform a process comprising:
- determining the user is looking at the one hand using image recognition and hand tracking;
- generating a digital model of the one hand by converting captured data on position, orientation, and velocity of the one hand in response to determining the user is looking at the one hand;
- identifying a position on the one hand from the digital model;
- projecting a virtual trackpad at the position on the one hand of a user using hand tracking position data from an extend reality (XR) device in response to determining the user is looking at the one hand;
- detecting, via hand tracking, movements of one or more fingers as the one or more fingers are being moved across the virtual trackpad projected on the one hand;
- and
- controlling a user interface (UI) being displayed on a display by the XR device based on the movements of the one or more fingers.
One or more non-transitory computer readable storage media having instructions stored thereupon which, when executed by a system having at least a processor and a memory therein, cause the system to perform operations comprising:
- determining the user is looking at the one hand using image recognition and hand tracking;
- generating a digital model of the one hand by converting captured data on position, orientation, and velocity of the one hand in response to determining the user is looking at the one hand;
- identifying a position on the one hand from the digital model;
- projecting a virtual trackpad at the position on the one hand of a user using hand tracking position data from an extend reality (XR) device in response to determining the user is looking at the one hand;
- detecting, via hand tracking, movements of one or more fingers as the one or more fingers are being moved across the virtual trackpad projected on the one hand;
- and
- controlling a user interface (UI) being displayed on a display by the XR device based on the movements of the one or more fingers.
Dependencies are inferred from the claim text.