# US 2026/0064265 A1 · Dynamic extended reality user interface

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0064265 A1
- Document ID: US20260064265A1
- Status: Published application
- Kind: A1
- Application number: 19/304,482
- Filed: 2025-08-19
- Published: 2026-03-05
- Earliest claimed: 2024-09-05
- Assignees: Snap Inc.
- Applicants: Snap Inc.
- Inventors: Mitchell Kuppersmith; Karen Stolzenberg; Brian Wong
- Web page: https://patentgrep.com/patent/US20260064265A1

## CPC

- G06F 3/04886: by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
- G02B 27/0101: {characterised by optical features (G02B27/0172 takes precedence)}
- G02B 27/017: {Head mounted}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)}
- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06F 40/40: Processing or translation of natural language (natural language analysis G06F40/20; semantic analysis G06F40/30)
- G06V 20/20: in augmented reality scenes
- G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)}
- G02B 2027/0138: {comprising image capture systems, e.g. camera}
- G02B 2027/014: {comprising information/image processing systems}
- G02B 2027/0178: {Eyeglass type (eyeglass details G02C)}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

An extended Reality (XR) system is provided that generates a dynamic XR user interface having a variety of user input modalities and types of XR user interfaces. The XR system provides a body-centric XR user interface on a hand of the user including a first interactive virtual object located on the hand. The XR system detects a first selection of the first interactive virtual object and provides a near-field XR user interface including a second interactive virtual object. The XR system detects a second selection of the second interactive virtual object and configures the near-field XR user interface to capture a user input. The XR user interface, captures the user input using the near-field XR user interface, generates content for a far-field XR user interface, provides the far-field XR user interface to the user, and displays the content to the user using the far-field XR user interface.

## Claims

### Claim 1

A machine-implemented method, comprising:
providing, to a user of an extended Reality (XR) system, a body-centric XR user interface on a hand of the user outside of a field of view of the user, the body-centric XR user interface including an interactive virtual object located on the hand;
capturing, using one or more tracking sensors of the XR system, tracking data of the hand of the user;
continuously updating an XR user interface object model with a current location and position of the hand and the interactive virtual object using the tracking data while the body-centric XR user interface remains outside the field of view of the user;
detecting a selection of the interactive virtual object by the user while the XR user interface is outside of the field of view of the user; and
in response to detecting the selection, providing a near-field XR user interface to the user within the field of view of the user.

### Claim 2 (depends on 1)

The machine-implemented method of claim 1, wherein the body-centric XR user interface is located on a first hand of the user, and wherein detecting the selection of the interactive virtual object comprises:
capturing, using one or more image sensors of the XR system, image data of the first hand and a second hand of the user; and
detecting a hand touch by a digit of the second hand at a location of the interactive virtual object on the first hand using the image data.

### Claim 3 (depends on 1)

The machine-implemented method of claim 1, wherein the interactive virtual object is provided in association with a specified portion of a palmar surface of the hand, and wherein the user uses proprioception to touch the specified portion of the palmar surface at the location that corresponds to the interactive virtual object.

### Claim 4 (depends on 3)

The machine-implemented method of claim 3, wherein the specified portion of the palmar surface comprises one of a thenar eminence at a thumb base, a hypothenar eminence at a little finger side of the palmar surface, or one or more interdigital spaces between fingers.

### Claim 5 (depends on 1)

The machine-implemented method of claim 1, wherein the one or more tracking sensors comprise one or more cameras that have a wide field of view and capture images of the hand of the user when the hand is out of the field of view of the user.

### Claim 6 (depends on 1)

The machine-implemented method of claim 1, wherein continuously updating the XR user interface object model comprises determining that the interactive virtual object is outside of the field of view of the user and not rendering the interactive virtual object while tracking the current location and position of the interactive virtual object.

### Claim 7 (depends on 1)

The machine-implemented method of claim 1, wherein the XR system is a head-wearable apparatus.

### Claim 8

A machine comprising:
at least one processor; and
at least one memory storing instructions that, when executed by the at least one processor, cause the machine to perform operations comprising:
providing, to a user of an extended Reality (XR) system, a body-centric XR user interface on a hand of the user outside of a field of view of the user, the body-centric XR user interface including an interactive virtual object located on the hand;
capturing, using one or more tracking sensors of the XR system, tracking data of the hand of the user;
continuously updating an XR user interface object model with a current location and position of the hand and the interactive virtual object using the tracking data while the body-centric XR user interface remains outside the field of view of the user;
detecting a selection of the interactive virtual object by the user while the XR user interface is outside of the field of view of the user; and
in response to detecting the selection, providing a near-field XR user interface to the user within the field of view of the user.

### Claim 9 (depends on 8)

The machine of claim 8, wherein the body-centric XR user interface is located on a first hand of the user, and wherein detecting the selection of the interactive virtual object comprises:
capturing, using one or more image sensors of the XR system, image data of the first hand and a second hand of the user; and
detecting a hand touch by a digit of the second hand at a location of the interactive virtual object on the first hand using the image data.

### Claim 10 (depends on 8)

The machine of claim 8, wherein the interactive virtual object is provided in association with a specified portion of a palmar surface of the hand, and wherein the user uses proprioception to touch the specified portion of the palmar surface at the location that corresponds to the interactive virtual object.

### Claim 11 (depends on 10)

The machine of claim 10, wherein the specified portion of the palmar surface comprises one of a thenar eminence at a thumb base, a hypothenar eminence at a little finger side of the palmar surface, or one or more interdigital spaces between fingers.

### Claim 12 (depends on 8)

The machine of claim 8, wherein the one or more tracking sensors comprise one or more cameras that have a wide field of view and capture images of the hand of the user when the hand is out of the field of view of the user.

### Claim 13 (depends on 8)

The machine of claim 8, wherein continuously updating the XR user interface object model comprises determining that the interactive virtual object is outside of the field of view of the user and not rendering the interactive virtual object while tracking the current location and position of the interactive virtual object.

### Claim 14 (depends on 8)

The machine of claim 8, wherein the XR system is a head-wearable apparatus.

### Claim 15

A machine-storage medium including instructions that, when executed by a machine, cause the machine to perform operations comprising:
providing, to a user of an extended Reality (XR) system, a body-centric XR user interface on a hand of the user outside of a field of view of the user, the body-centric XR user interface including an interactive virtual object located on the hand;
capturing, using one or more tracking sensors of the XR system, tracking data of the hand of the user;
continuously updating an XR user interface object model with a current location and position of the hand and the interactive virtual object using the tracking data while the body-centric XR user interface remains outside the field of view of the user;
detecting a selection of the interactive virtual object by the user while the XR user interface is outside of the field of view of the user; and
in response to detecting the selection, providing a near-field XR user interface to the user within the field of view of the user.

### Claim 16 (depends on 15)

The machine-storage medium of claim 15, wherein the body-centric XR user interface is located on a first hand of the user, and wherein detecting the selection of the interactive virtual object comprises:
capturing, using one or more image sensors of the XR system, image data of the first hand and a second hand of the user; and
detecting a hand touch by a digit of the second hand at a location of the interactive virtual object on the first hand using the image data.

### Claim 17 (depends on 15)

The machine-storage medium of claim 15, wherein the interactive virtual object is provided in association with a specified portion of a palmar surface of the hand, and wherein the user uses proprioception to touch the specified portion of the palmar surface at the location that corresponds to the interactive virtual object.

### Claim 18 (depends on 17)

The machine-storage medium of claim 17, wherein the specified portion of the palmar surface comprises one of a thenar eminence at a thumb base, a hypothenar eminence at a little finger side of the palmar surface, or one or more interdigital spaces between fingers.

### Claim 19 (depends on 15)

The machine-storage medium of claim 15, wherein the one or more tracking sensors comprise one or more cameras that have a wide field of view and capture images of the hand of the user when the hand is out of the field of view of the user.

### Claim 20 (depends on 15)

The machine-storage medium of claim 15, wherein the XR system is a head-wearable apparatus.
