# US 2026/0064189 A1 · Dynamic extended reality user interface

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0064189 A1
- Document ID: US20260064189A1
- Status: Published application
- Kind: A1
- Application number: 18/826,091
- Filed: 2024-09-05
- Published: 2026-03-05
- Earliest claimed: 2024-09-05
- Assignees: Snap Inc.
- Applicants: Snap Inc.
- Inventors: Mitchell Kuppersmith; Karen Stolzenberg; Brian Wong
- Web page: https://patentgrep.com/patent/US20260064189A1

## CPC

- G06F 3/04886: by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
- G02B 27/0101: {characterised by optical features (G02B27/0172 takes precedence)}
- G02B 27/017: {Head mounted}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)}
- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06F 40/40: Processing or translation of natural language (natural language analysis G06F40/20; semantic analysis G06F40/30)
- G06V 20/20: in augmented reality scenes
- G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)}
- G02B 2027/0138: {comprising image capture systems, e.g. camera}
- G02B 2027/014: {comprising information/image processing systems}
- G02B 2027/0178: {Eyeglass type (eyeglass details G02C)}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

An extended Reality (XR) system is provided that generates a dynamic XR user interface having a variety of user input modalities and types of XR user interfaces. The XR system provides a body-centric XR user interface on a hand of the user including a first interactive virtual object located on the hand. The XR system detects a first selection of the first interactive virtual object and provides a near-field XR user interface including a second interactive virtual object. The XR system detects a second selection of the second interactive virtual object and configures the near-field XR user interface to capture a user input. The XR user interface, captures the user input using the near-field XR user interface, generates content for a far-field XR user interface, provides the far-field XR user interface to the user, and displays the content to the user using the far-field XR user interface.

## Claims

### Claim 1

A machine-implemented method, comprising:
causing display of a body-centric extended Reality (XR) user interface on a hand of a user of an XR system, the body-centric XR user interface including a first interactive virtual object displayed on the hand;
detecting a first selection of the first interactive virtual object; and
in response to detecting the first selection of the first interactive virtual object, performing first operations comprising:
causing display of a near-field XR user interface including a second interactive virtual object;
detecting a second selection of the second interactive virtual object; and
in response to detecting the second selection, performing second operations comprising:
configuring the near-field XR user interface to capture a user input based on the interactive virtual object;
capturing the user input using the near-field XR user interface; and
in response to capturing the user input, performing third operations comprising:
generating content for a far-field XR user interface using the user input; and
causing display of the generated content in the far-field XR user interface.

### Claim 2 (depends on 1)

The machine-implemented method of claim 1, wherein before causing display of the body-centric XR user interface, the computer-implemented method comprises:
capturing, using one or more tracking sensors of the XR system, tracking data of the hand of the user;
detecting a palm-up gesture of the hand using the tracking data; and
in response to detecting the palm-up gesture, causing display of the body-centric XR user interface.

### Claim 3 (depends on 1)

The machine-implemented method of claim 1,
wherein the body-centric XR user interface is located on a first hand of the user, and
wherein detecting the first selection of the first interactive virtual object comprises:
capturing, using one or more image sensors of the XR system, image data of the first hand and a second hand of the user; and
detecting a hand touch by a digit of the second hand at the location of the first interactive virtual object on the first hand using the image data.

### Claim 4 (depends on 1)

The machine-implemented method of claim 1, wherein the near-field user interface is configured to detect the second selection of the second interactive virtual object using a Direct Manipulation of Virtual Object (DMVO) user input modality.

### Claim 5 (depends on 1)

The machine-implemented method of claim 1,
wherein the near-field user interface is configured to capture speech data, and
wherein capturing the user input comprises capturing speech data from the user.

### Claim 6 (depends on 5)

The machine-implemented method of claim 5, wherein generating content for the far-field XR user interface using the input data comprises:
generating prompt data for a generative model using the speech data;
prompting the generative model using the prompt data; and
receiving the content from the generative model.

### Claim 7 (depends on 1)

The machine-implemented method of claim 1,
wherein the near-field user interface is configured to capture image data, and
wherein capturing the user input comprises:
capturing, using one or more tracking sensors of the XR system, tracking data of one or more hands of the user;
recognizing a hand gesture using the tracking data; and
in response to recognizing the hand gesture, capturing, using the one or more tracking sensors of the XR system, image data of a real-world environment in a field of view of the user.

### Claim 8 (depends on 7)

The machine-implemented method of claim 7,
wherein the near-field user interface is configured to capture image data in response to the user interacting with a third interactive virtual object using a DMVO user input modality, and
wherein capturing the user input comprises:
capturing, using one or more tracking sensors of the XR system, tracking data of one or more hands of the user;
detecting a third selection by the user of the third interactive virtual object; and
in response to detecting the third selection, capturing, using the one or more tracking sensors of the XR system, image data of a real-world environment in a field of view of the user.

### Claim 9 (depends on 1)

The machine-implemented method of claim 1, wherein the XR system is a head-wearable apparatus.

### Claim 10

A machine comprising:
at least one processor; and
at least one memory storing instructions that, when executed by the at least one processor, cause the machine to perform operations comprising:
causing display of a body-centric extended Reality (XR) user interface on a hand of a user of an XR system, the body-centric XR user interface including a first interactive virtual object displayed on the hand;
detecting a first selection of the first interactive virtual object; and
in response to detecting the first selection of the first interactive virtual object, performing first operations comprising:
causing display of a near-field XR user interface including a second interactive virtual object;
detecting a second selection of the second interactive virtual object; and
in response to detecting the second selection, performing second operations comprising:
configuring the near-field XR user interface to capture a user input based on the interactive virtual object;
capturing the user input using the near-field XR user interface; and
in response to capturing the user input, performing third operations comprising:
generating content for a far-field XR user interface using the user input; and
causing display of the generated content in the far-field XR user interface.

### Claim 11 (depends on 10)

The machine of claim 10, wherein before causing display of the body-centric XR user interface, the computer-implemented method comprises:
capturing, using one or more tracking sensors of the XR system, tracking data of the hand of the user;
detecting a palm-up gesture of the hand using the tracking data; and
in response to detecting the palm-up gesture, causing display of the body-centric XR user interface.

### Claim 12 (depends on 10)

The machine of claim 10,
wherein the body-centric XR user interface is located on a first hand of the user, and
wherein detecting the first selection of the first interactive virtual object comprises:
capturing, using one or more image sensors of the XR system, image data of the first hand and a second hand of the user; and
detecting a hand touch by a digit of the second hand at the location of the first interactive virtual object on the first hand using the image data.

### Claim 13 (depends on 10)

The machine of claim 10, wherein the near-field user interface is configured to detect the second selection of the second interactive virtual object using a Direct Manipulation of Virtual Object (DMVO) user input modality.

### Claim 14 (depends on 10)

The machine of claim 10,
wherein the near-field user interface is configured to capture speech data, and
wherein capturing the user input comprises capturing speech data from the user.

### Claim 15 (depends on 14)

The machine of claim 14, wherein generating content for the far-field XR user interface using the input data comprises:
generating prompt data for a generative model using the speech data;
prompting the generative model using the prompt data; and
receiving the content from the generative model.

### Claim 16 (depends on 10)

The machine of claim 10,
wherein the near-field user interface is configured to capture image data, and
wherein capturing the user input comprises:
capturing, using one or more tracking sensors of the XR system, tracking data of one or more hands of the user;
recognizing a hand gesture using the tracking data; and
in response to recognizing the hand gesture, capturing, using the one or more tracking sensors of the XR system, image data of a real-world environment in a field of view of the user.

### Claim 17 (depends on 16)

The machine of claim 16,
wherein the near-field user interface is configured to capture image data in response to the user interacting with a third interactive virtual object using a DMVO user input modality, and
wherein capturing the user input comprises:
capturing, using one or more tracking sensors of the XR system, tracking data of one or more hands of the user;
detecting a third selection by the user of the third interactive virtual object; and
in response to detecting the third selection, capturing, using the one or more tracking sensors of the XR system, image data of a real-world environment in a field of view of the user.

### Claim 18 (depends on 10)

The machine of claim 10, wherein the XR system is a head-wearable apparatus.

### Claim 19

A machine-storage medium, the machine-storage medium including instructions that, when executed by a machine, cause the machine to perform operations comprising:
causing display of a body-centric extended Reality (XR) user interface on a hand of a user of an XR system, the body-centric XR user interface including a first interactive virtual object displayed on the hand;
detecting a first selection of the first interactive virtual object; and
in response to detecting the first selection of the first interactive virtual object, performing first operations comprising:
causing display of a near-field XR user interface including a second interactive virtual object;
detecting a second selection of the second interactive virtual object; and
in response to detecting the second selection, performing second operations comprising:
configuring the near-field XR user interface to capture a user input based on the interactive virtual object;
capturing the user input using the near-field XR user interface; and
in response to capturing the user input, performing third operations comprising:
generating content for a far-field XR user interface using the user input; and
causing display of the generated content in the far-field XR user interface.

### Claim 20 (depends on 19)

The machine-storage medium of claim 19, wherein before causing display of the body-centric XR user interface, the computer-implemented method comprises:
capturing, using one or more tracking sensors of the XR system, tracking data of the hand of the user;
detecting a palm-up gesture of the hand using the tracking data; and
in response to detecting the palm-up gesture, causing display of the body-centric XR user interface.
