# US 2026/0310331 A1 · Extended reality hand-located user interfaces

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0310331 A1
- Document ID: US20260310331A1
- Status: Published application
- Kind: A1
- Application number: 19/715,378
- Filed: 2026-06-22
- Published: 2026-10-08
- Earliest claimed: 2024-09-05
- Assignees: Snap Inc.
- Applicants: Snap Inc.
- Inventors: Viktoria Hwang; James Powderly; Evan Spiegel; Karen Stolzenberg; Mathieu Emmanuel Vignau
- Web page: https://patentgrep.com/patent/US20260310331A1

## CPC

- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

An eXtended Reality (XR) system that provides a hand-located XR user interface to a user. The XR system captures tracking data of a user's hands using one or more sensors. The XR system uses the tracking data to generate the hand-located XR user interface that includes one or more interactive virtual objects associated with one or more locations on a surface of the first hand of the user which the user can interact with using a digit of a second hand. Upon detecting a hand touch of the surface of the first hand by a digit of the second hand, the XR system executes a function or process associated with an interactive virtual object associated with the location of the hand touch. The hand-located XR user interface can be provided within or outside of a field of view of an optical element that provides images of the interactive virtual objects.

## Claims

### Claim 1

A machine-implemented method, comprising:
capturing, using one or more sensors of an Extended Reality (XR) system, tracking data of a first hand and a second hand of a user; and
while continuously capturing the tracking data, performing operations comprising:
generating, using the tracking data, a hand-located user interface including an interactive virtual object associated with a location on a palmar surface of the first hand, the location being a fleshy portion of the palmar surface;
providing the hand-located user interface to the user;
detecting, using the tracking data, a hand touch on the palmar surface of the first hand by a digit of the second hand at the location on the first hand; and
performing an action by the XR system based on the interactive virtual object associated with the location on the first hand.

### Claim 2 (depends on 1)

The machine-implemented method of claim 1, wherein detecting the hand touch on the palmar surface of the first hand by the digit of the second hand at the location further comprises:
detecting, using the tracking data, a deformation formed in a fleshy part of the palmar surface at the location by the digit of the second hand pressing against the palmar surface.

### Claim 3 (depends on 1)

The machine-implemented method of claim 1, wherein the hand-located user interface includes one or more interactive virtual objects provided in association with one or more respective locations of the palmar surface of the first hand.

### Claim 4 (depends on 3)

The machine-implemented method of claim 3, wherein the one or more respective locations on the palmar surface of the first hand include a thenar eminence of the first hand.

### Claim 5 (depends on 3)

The machine-implemented method of claim 3, wherein the one or more respective locations on the palmar surface of the first hand include a hypothenar eminence of the first hand.

### Claim 6 (depends on 1)

The machine-implemented method of claim 1, wherein the one or more respective locations on the palmar surface of the first hand include an interdigital space of the first hand.

### Claim 7 (depends on 1)

The machine-implemented method of claim 1, wherein the XR system comprises a head-wearable apparatus.

### Claim 8

A machine comprising:
at least one processor; and
at least one memory storing instructions that, when executed by the at least one processor, cause the machine to perform operations comprising:
capturing, using one or more sensors of an Extended Reality (XR) system, tracking data of a first hand and a second hand of a user; and
while continuously capturing the tracking data, performing operations comprising:
generating, using the tracking data, a hand-located user interface including an interactive virtual object associated with a location on a palmar surface of the first hand, the location being a fleshy portion of the palmar surface;
providing the hand-located user interface to the user;
detecting, using the tracking data, a hand touch on the palmar surface of the first hand by a digit of the second hand at the location on the first hand; and
performing an action by the XR system based on the interactive virtual object associated with the location on the first hand.

### Claim 9 (depends on 8)

The machine of claim 8, wherein detecting the hand touch on the palmar surface of the first hand by the digit of the second hand at the location further comprises:
detecting, using the tracking data, a deformation formed in a fleshy part of the palmar surface at the location by the digit of the second hand pressing against the palmar surface.

### Claim 10 (depends on 8)

The machine of claim 8, wherein the hand-located user interface includes one or more interactive virtual objects provided in association with one or more respective locations of the palmar surface of the first hand.

### Claim 11 (depends on 10)

The machine of claim 10, wherein the one or more respective locations on the palmar surface of the first hand include a thenar eminence of the first hand.

### Claim 12 (depends on 10)

The machine of claim 10, wherein the one or more respective locations on the palmar surface of the first hand include a hypothenar eminence of the first hand.

### Claim 13 (depends on 8)

The machine of claim 8, wherein the one or more respective locations on the palmar surface of the first hand include an interdigital space of the first hand.

### Claim 14 (depends on 8)

The machine of claim 8, wherein the XR system comprises a head-wearable apparatus.

### Claim 15

A machine-storage medium including instructions that, when executed by a machine, cause the machine to perform operations comprising:
capturing, using one or more sensors of an Extended Reality (XR) system, tracking data of a first hand and a second hand of a user; and
while continuously capturing the tracking data, performing operations comprising:
generating, using the tracking data, a hand-located user interface including an interactive virtual object associated with a location on a palmar surface of the first hand, the location being a fleshy portion of the palmar surface;
providing the hand-located user interface to the user;
detecting, using the tracking data, a hand touch on the palmar surface of the first hand by a digit of the second hand at the location on the first hand; and
performing an action by the XR system based on the interactive virtual object associated with the location on the first hand.

### Claim 16 (depends on 15)

The machine-storage medium of claim 15, wherein detecting the hand touch on the palmar surface of the first hand by the digit of the second hand at the location further comprises:
detecting, using the tracking data, a deformation formed in a fleshy part of the palmar surface at the location by the digit of the second hand pressing against the palmar surface.

### Claim 17 (depends on 15)

The machine-storage medium of claim 15, wherein the hand-located user interface includes one or more interactive virtual objects provided in association with one or more respective locations of the palmar surface of the first hand.

### Claim 18 (depends on 17)

The machine-storage medium of claim 17, wherein the one or more respective locations on the palmar surface of the first hand include a thenar eminence of the first hand.

### Claim 19 (depends on 17)

The machine-storage medium of claim 17, wherein the one or more respective locations on the palmar surface of the first hand include a hypothenar eminence of the first hand.

### Claim 20 (depends on 15)

The machine-storage medium of claim 15, wherein the one or more respective locations on the palmar surface of the first hand include an interdigital space of the first hand.
