Augmented reality hand gesture recognition systems

A method, comprising:

• detecting, by a sensor, a position of a first digit of a hand of a user;
• detecting, by the sensor, a position of a second digit of the hand of the user;
• detecting, by the sensor, a position of a wrist or a palm of the hand of the user;
• identifying, by a processing device, a first fingertip of the first digit;
• identifying, by the processing device, a second fingertip of the second digit;
• identifying a wrist point of the wrist or a palm-center of the palm;
• outputting, to a display device, data for displaying a virtual object;
• in response to determining the orientation of the wrist point or the palm-center is towards the virtual object, outputting, to the display device, data for displaying the cursor at a virtual position of the virtual object;
• determining, by the processing device, that the first fingertip and the second fingertip are in an open pinch position of a pinch gesture at a first point in time;
• determining, by the processing device, that the first fingertip and the second fingertip are in a closed pinch position of a pinch gesture at a second point in time, wherein the second point in time is after the first point in time, the wherein determining that the first fingertip and the second fingertip are in the closed position comprises:
• comparing a current position of the first fingertip and a current position of the second fingertip to a dataset of average positions for the closed pinch position of the first digit and the second digit given a palm size of the user;
• determining an average distance between the first digit and the second digit for the closed pinch position based on the dataset of average positions;
• and
• defining a buffer value for the closed pinch position;
• designating the first fingertip and the second fingertip as being in the closed pinch position when a distance between the first digit and the second digit is less than the average distance less the buffer value;
• determining, by the processing device, that the first fingertip and the second fingertip are in the closed pinch position for a threshold period of time;
• determining, in response to detection of the pinch gesture, a virtual position of the cursor at the virtual position of the virtual object based on a position of a palm center and/or wrist point during the pinch gesture, wherein the virtual position of the cursor ignores the position of the first fingertip of the first digit and the second fingertip of the second digit during the pinch gesture;
• and
• executing, at an end of the threshold period of time, a tap instruction associated with the closed pinch position.

A device, comprising:

• a wearable display configured to:
• attach to a head of a user;
• and
• display an augmented reality environment to the user, wherein the augmented reality environment includes a virtual object;
• a sensor configured to:
• detect a position of a first digit of a hand of the user;
• and
• detect a position of a second digit of the hand of the user;
• detect a position of a palm center and/or wrist point of the hand of the user;
• and
• a processing device coupled to the wearable display and the sensor, wherein the processing device is configured to:
• define a tap instruction that is associated with the virtual object;
• map a closed pinch position to the tap instruction such that detection of the closed pinch position by the sensor triggers execution of the tap instruction by the processing device;
• output, to the wearable display, data for displaying the virtual object;
• identify a first fingertip of the first digit;
• identify a second fingertip of the second digit;
• determine that the first fingertip and the second fingertip are in a closed pinch position of a pinch gesture for a first threshold period of time, the determination that the first fingertip and the second fingertip are in the closed position comprises:
• compare a current position of the first fingertip and a current position of the second fingertip to a dataset of average positions for the closed pinch position of the first digit and the second digit given a palm size of the user;
• determine an average distance between the first digit and the second digit for the closed pinch position based on the dataset of average positions;
• and
• define a buffer value for the closed pinch position;
• designate the first fingertip and the second fingertip as being in the closed pinch position when a distance between the first digit and the second digit is less than the average distance less the buffer value;
• output, in response to a detection of the pinch gesture, a virtual position of the cursor based on the position of the palm center and/or the wrist point of the hand of the user during the pinch gesture to exclusion of the position of the first fingertip of the first digit and the second fingertip of the second digit during the pinch gesture;
• and
• execute the tap instruction at an end of the first threshold period of time.