# US 2026/0064237 A1 · Extended reality hand-located user interfaces

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0064237 A1
- Document ID: US20260064237A1
- Status: Published application
- Kind: A1
- Application number: 18/825,800
- Filed: 2024-09-05
- Published: 2026-03-05
- Earliest claimed: 2024-09-05
- Assignees: Snap Inc.
- Applicants: Snap Inc.
- Inventors: Viktoria Hwang; James Powderly; Evan Spiegel; Karen Stolzenberg; Mathieu Emmanuel Vignau
- Web page: https://patentgrep.com/patent/US20260064237A1

## CPC

- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

An extended Reality (XR) system that provides a hand-located XR user interface to a user. The XR system captures tracking data of a user's hands using one or more sensors. The XR system uses the tracking data to generate the hand-located XR user interface that includes one or more interactive virtual objects associated with one or more locations on a surface of the first hand of the user which the user can interact with using a digit of a second hand. Upon detecting a hand touch of the surface of the first hand by a digit of the second hand, the XR system executes a function or process associated with an interactive virtual object associated with the location of the hand touch. The hand-located XR user interface can be provided within or outside of a field of view of an optical element that provides images of the interactive virtual objects.

## Claims

### Claim 1

A machine-implemented method, comprising:
capturing, using one or more sensors of an Extended Reality (XR) system, tracking data of a first hand and a second hand of a user; and
while continuously capturing the tracking data, performing operations comprising:
generating, using the tracking data, a hand-located proprioceptive user interface including an interactive virtual object associated with a location on a surface of the first hand, the hand-located proprioceptive user interface provided to the user outside of an optical element field of view of one or more optical elements of the XR system;
providing the hand-located proprioceptive user interface to the user;
detecting, using the tracking data, a hand touch on the surface of the first hand by a digit of the second hand at the location on the first hand; and
performing an action by the XR system based on the interactive virtual object associated with the location on the first hand.

### Claim 2 (depends on 1)

The machine-implemented method of claim 1, wherein the surface of the first hand is a palmar surface.

### Claim 3 (depends on 2)

The machine-implemented method of claim 2, wherein generating, using the tracking data, the hand-located user interface further comprises:
detecting, using the tracking data, a first being made by the first hand;
detecting, using the tracking data, the first being opened up exposing the palmar surface such that the palmar surface is facing upward; and
in response to detecting the first being opened, initiating the generating of the hand-located user interface on the palmar surface of the first hand.

### Claim 4 (depends on 1)

The machine-implemented method of claim 1, wherein the surface of the first hand is a hand dorsal surface.

### Claim 5 (depends on 4)

The machine-implemented method of claim 4, wherein generating, using the tracking data, the hand-located user interface further comprises:
detecting, using the tracking data, the first hand positioned such that the hand dorsal surface faces upward and one or more fingers of the first hand are extended; and
in response to detecting the first hand positioned such that the hand dorsal surface faces upward and one or more fingers of the first hand are extended, initiating the generating of the hand-located user interface on the hand dorsal surface of the first hand.

### Claim 6

(canceled)

### Claim 7 (depends on 1)

The machine-implemented method of claim 1, wherein the XR system is a head-wearable apparatus.

### Claim 8

A machine comprising:
at least one processor; and
at least one memory storing instructions that, when executed by the at least one processor, cause the machine to perform operations comprising:
capturing, using one or more sensors of an Extended Reality (XR) system, tracking data of a first hand and a second hand of a user; and
while continuously capturing the tracking data, performing operations comprising:
generating, using the tracking data, a hand-located proprioceptive user interface including an interactive virtual object associated with a location on a surface of the first hand, the hand-located proprioceptive user interface provided to the user outside of an optical element field of view of one or more optical elements of the XR system;
providing the hand-located proprioceptive user interface to the user;
detecting, using the tracking data, a hand touch on the surface of the first hand by a digit of the second hand at the location on the first hand; and
performing an action by the XR system based on the interactive virtual object associated with the location on the first hand.

### Claim 9 (depends on 8)

The machine of claim 8, wherein the surface of the first hand is a palmar surface.

### Claim 10 (depends on 9)

The machine of claim 9, wherein generating, using the tracking data, the hand-located user interface further comprises:
detecting, using the tracking data, a first being made by the first hand;
detecting, using the tracking data, the first being opened up exposing the palmar surface such that the palmar surface is facing upward; and
in response to detecting the first being opened, initiating the generating of the hand-located user interface on the palmar surface of the first hand.

### Claim 11 (depends on 8)

The machine of claim 8, wherein the surface of the first hand is a hand dorsal surface.

### Claim 12 (depends on 11)

The machine of claim 11, wherein generating, using the tracking data, the hand-located user interface further comprises:
detecting, using the tracking data, the first hand positioned such that the hand dorsal surface faces upward and one or more fingers of the first hand are extended; and
in response to detecting the first hand positioned such that the hand dorsal surface faces upward and one or more fingers of the first hand are extended, initiating the generating of the hand-located user interface on the hand dorsal surface of the first hand.

### Claim 13

(canceled)

### Claim 14 (depends on 8)

The machine of claim 8, wherein the XR system is a head-wearable apparatus.

### Claim 15

A machine-storage medium, the machine-storage medium including instructions that, when executed by a machine, cause the machine to perform operations comprising:
capturing, using one or more sensors of an Extended Reality (XR) system, tracking data of a first hand and a second hand of a user, and
while continuously capturing the tracking data, performing operations comprising:
generating, using the tracking data, a hand-located proprioceptive user interface including an interactive virtual object associated with a location on a surface of the first hand, the hand-located proprioceptive user interface provided to the user outside of an optical element field of view of one or more optical elements of the XR system;
providing the hand-located proprioceptive user interface to the user;
detecting, using the tracking data, a hand touch on the surface of the first hand by a digit of the second hand at the location on the first hand; and
performing an action by the XR system based on the interactive virtual object associated with the location on the first hand.

### Claim 16 (depends on 15)

The machine-storage medium of claim 15, wherein the surface of the first hand is a palmar surface.

### Claim 17 (depends on 16)

The machine-storage medium of claim 16, wherein generating, using the tracking data, the hand-located user interface further comprises:
detecting, using the tracking data, a first being made by the first hand;
detecting, using the tracking data, the first being opened up exposing the palmar surface such that the palmar surface is facing upward; and
in response to detecting the first being opened, initiating the generating of the hand-located user interface on the palmar surface of the first hand.

### Claim 18 (depends on 15)

The machine-storage medium of claim 15, wherein the surface of the first hand is a hand dorsal surface.

### Claim 19 (depends on 18)

The machine-storage medium of claim 18, wherein generating, using the tracking data, the hand-located user interface further comprises:
detecting, using the tracking data, the first hand positioned such that the hand dorsal surface faces upward and one or more fingers of the first hand are extended; and
in response to detecting the first hand positioned such that the hand dorsal surface faces upward and one or more fingers of the first hand are extended, initiating the generating of the hand-located user interface on the hand dorsal surface of the first hand.

### Claim 20 (depends on 15)

The machine-storage medium of claim 15, wherein the XR system is a head-wearable apparatus.
