# US 2026/0016951 A1 · Electronic device for supporting xr content, and input mode support method therefor

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0016951 A1
- Document ID: US20260016951A1
- Status: Published application
- Kind: A1
- Application number: 19/334,330
- Filed: 2025-09-19
- Published: 2026-01-15
- Earliest claimed: 2023-03-21
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Sungkwang YANG; Kwangtai KIM; Seungnyun KIM; Jaekyu SHIM; Jongmin YOON; Joungmin CHO
- Web page: https://patentgrep.com/patent/US20260016951A1

## CPC

- G06F 3/04883: for inputting data by handwriting, e.g. gesture or text
- G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking}
- G02B 27/01: Head-up displays
- G02B 27/0172: {characterised by optical features}
- G06F 3/00: Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F 3/01: Input arrangements or combined input and output arrangements for interaction between user and computer (G06F3/16 takes precedence)
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/012: {Head tracking input arrangements}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)}
- G06F 3/041: Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F 3/0481: based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics
- G06V 40/18: Eye characteristics, e.g. of the iris
- G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16)
- G02B 2027/0138: {comprising image capture systems, e.g. camera}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

An electronic device includes: a body; a display facing a first direction; a touch sensor facing in a second direction; first cameras facing the second direction; at least one processor; and memory storing instructions, wherein the instructions, when executed by the at least one processor individually or collectively, to cause the electronic device to: control the display to output an extended reality (XR) content screen including at least one virtual object; based on identifying that a hand of the user is within a first distance from the touch sensor, control the display to change the hand of the user displayed in the XR content screen to a virtual pointer object; based on a touch input being detected via the touch sensor, control the display to display a virtual area frame including the virtual pointer object; and control motion of the virtual pointer object in the virtual area frame.

## Claims

### Claim 1

An electronic device comprising:
a body;
a display provided at a first side of the body and facing a first direction toward a user of the electronic device;
a touch sensor provided at a second side of the body and facing in a second direction opposite to the first direction of the body;
a plurality of first cameras provided at the second side of the body and facing the second direction;
at least one processor inside the body, and operatively connected to the display, the touch sensor, and the plurality of first cameras; and
memory storing instructions,
wherein the instructions, when executed by the processor individually or collectively, to cause the electronic device to:
control the display to output an extended reality (XR) content screen including at least one virtual object;
based on identifying that a hand of the user is within a first distance from the touch sensor based on information obtained through at least one of the touch sensor or the plurality of first cameras, control the display to change the hand of the user displayed in the XR content screen to a virtual pointer object displayed in the XR content screen;
based on a touch input being detected via the touch sensor, control the display to display a virtual area frame including the virtual pointer object in the XR content screen, the virtual area frame having a size corresponding to a touch sensor area of the touch sensor; and
control motion of the virtual pointer object in the virtual area frame through a motion vector designated in correspondence to a touch input vector detected via the touch sensor.

### Claim 2 (depends on 1)

The electronic device of claim 1, wherein the electronic device is a head-mounted display device, and
wherein the XR content screen comprises at least one screen providing a virtual reality (VR) service, and augmented reality (AR) service, or a mixed reality (MR) service.

### Claim 3 (depends on 1)

The electronic device of claim 1, wherein the plurality of first cameras comprises:
a first functional camera configured to support at least one of head tracking, hand detection, hand tracking, or spatial recognition;
a second functional camera configured to capture an outside environment and generate an image or video corresponding to the outside environment; and
at least one of a depth sensor or depth camera configured to measure a distance between an external object and the electronic device, and
wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to:
based on identifying that the hand of the user is positioned at a distance farther than the first distance from the touch sensor based on information obtained through at least one of the first functional camera and the depth sensor, display the XR content screen on the display in a first mode in which a hand-shaped object corresponding to the hand of the user is displayed in the XR content screen;
based on identifying that the hand of the user is positioned within the first distance, display the XR content screen on the display in a second mode in which the hand-shaped object is changed to the virtual pointer object; and
based on detection of a touch input via the touch sensor, displaying the XR content screen on the display in a third mode in which the virtual area frame is displayed in the XR content screen.

### Claim 4 (depends on 3)

The electronic device of claim 3, wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to:
determine a motion vector of a hand gesture based on switching to the second mode; and
control the motion of the virtual pointer object by increasing a movement distance of the motion vector of the hand gesture by a set magnification ratio.

### Claim 5 (depends on 3)

The electronic device of claim 3, wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to:
allocate the virtual area frame to at least a portion of the XR content screen when switching to the third mode; and
restrict motion of the virtual pointer object to a space within the allocated virtual area frame.

### Claim 6 (depends on 3)

The electronic device of claim 3, wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to:
display an entirety of the XR content screen in a plurality of grid areas having a N*M array of grid areas according to a plurality of settings based on switching to the second mode; and
display a first grid area corresponding to a position of the virtual pointer object, among the plurality of grid areas, as a highlighted grid area.

### Claim 7 (depends on 6)

The electronic device of claim 6, wherein the at least one virtual object comprises a plurality of virtual objects, and
the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to:
display each of the plurality of virtual objects in different grid areas among the plurality of grid areas based on the plurality of virtual objects being present in the first grid area in the second mode.

### Claim 8 (depends on 3)

The electronic device of claim 3, wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to:
change the size of the virtual area frame when a multi-touch gesture to expand or reduce the size of the virtual area frame is received, based on switching to the third mode; and
change a motion vector value of the virtual object by a magnification ratio corresponding to the changed size of the virtual area frame.

### Claim 9 (depends on 1)

The electronic device of claim 1, further comprising:
a second camera configured to track a gaze of the user and arranged in the first direction of the body,
wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to:
track the gaze of the user via the second camera; and
based on the hand of the use entering within the first distance spaced apart from the touch sensor, position the virtual pointer object at a central position of a direction of the tracked gaze of the user.

### Claim 10 (depends on 3)

The electronic device of claim 3, wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to:
display an inside of the virtual area frame as a grid having a N*M array of grid areas based on switching to the third mode.

### Claim 11 (depends on 10)

The electronic device of claim 10, wherein the at least one virtual object comprises a plurality of virtual objects, and
wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to:
based on the plurality of virtual objects being present in the virtual area frame in the third mode, enlarge a space of the virtual area frame by a ratio, to display the virtual area frame as overlapping at least a portion of the XR content screen.

### Claim 12 (depends on 1)

The electronic device of claim 1, wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to:
calibrate an error range for a touch input position detected by the touch sensor and a position of the virtual object output according to a gaze direction of the user.

### Claim 13

An method of an electronic device supporting XR content, the method comprising:
outputting an extended reality (XR) content screen including at least one virtual object via a display of the electronic device;
based on identifying that a hand of a user is within a first distance from a touch sensor of the electronic device based on information obtained through at least one of the touch sensor or a plurality of first cameras of the electronic device, controlling the display to change the hand of the user displayed in the XR content screen to a virtual pointer object displayed in the XR content screen;
based on a touch input being detected via the touch sensor, controlling the display to display a virtual area frame including the virtual pointer object in the XR content screen, the virtual area frame having a size corresponding to a touch sensor area of the touch sensor; and
controlling motion of the virtual pointer object in the virtual area frame through a motion vector designated in correspondence to a touch input vector detected via the touch sensor.

### Claim 14 (depends on 13)

The method of claim 13, wherein the displaying of the virtual pointer object further comprises:
tracking a gaze of the user via a second camera of the electronic device; and
positioning the virtual pointer object at a central position of a direction of the tracked gaze of the user based on the hand of the user being within the first distance spaced apart from the touch sensor.

### Claim 15 (depends on 13)

The method of claim 13, wherein the outputting of the extended reality (XR) content screen to the display further comprises:
based on identifying that the hand of the user is positioned at a distance farther than the first distance from the touch sensor, displaying the XR content screen on the display in a first mode in which a hand-shaped object corresponding to the hand of the user is displayed in the XR content screen;
based on the hand of the user entering within the first distance, to display the XR content screen on the display, displaying the XR content screen on the display in a second mode in which the hand-shaped object is changed to the virtual pointer object; and
based on detection of a touch input via the touch sensor, displaying the XR content screen on the display in a third mode in which the virtual area frame is displayed in the XR content screen.

### Claim 16 (depends on 15)

The method of claim 15, wherein the displaying the XR content screen on the display in a second mode further comprises:
determining a motion vector of a hand gesture; and
controlling the motion of the virtual pointer object by increasing a movement distance of the motion vector of the hand gesture by a set magnification ratio.

### Claim 17 (depends on 15)

The method of claim 15, wherein the displaying the XR content screen on the display in a second mode further comprises:
displaying an entirety of the XR content screen in a plurality of grid areas having a N*M array of grid areas according to a plurality of settings; and
displaying a first grid area corresponding to a position of the virtual pointer object, among the plurality of grid areas, as a highlighted grid area.

### Claim 18 (depends on 15)

The method of claim 15, wherein the displaying the XR content screen on the display in a third mode further comprises:
displaying an inside of the virtual area frame as a grid having a N*M array of grid areas.

### Claim 19 (depends on 13)

The method of claim 13, further comprising:
changing the size of the virtual area frame when a multi-touch gesture to expand or reduce the size of the virtual area frame is received; and
changing a motion vector value of the virtual object by a magnification ratio corresponding to the changed size of the virtual area frame.

### Claim 20 (depends on 13)

A non-transitory computer-readable recording media having recorded thereon a program that is executed by at least one processor of an electronic device to perform the method of claim 13.
