# US 2025/0155984 A1 · Virtual image display system and control method thereof

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0155984 A1
- Document ID: US20250155984A1
- Status: Published application
- Kind: A1
- Application number: 18/507,097
- Filed: 2023-11-13
- Published: 2025-05-15
- Earliest claimed: 2023-11-13
- Assignees: HTC Corporation
- Applicants: HTC Corporation
- Inventors: Su Kang Chou; Chih Chien Chen
- Web page: https://patentgrep.com/patent/US20250155984A1

## CPC

- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G02B 27/017: {Head mounted}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS

## Abstract

A virtual image display system and a control method thereof are provided. The virtual image display system includes a virtual image display device and an accessory. The accessory is disposed on a hand of a user and coupled to the virtual image display device. The virtual image display device is configured to: perform a gesture tracking action of the hand of the user according to positional information of the accessory; create a virtual pen and a virtual desktop according to gesture tracking information; decide whether to generate a virtual handwriting by determining a positional relationship between a pen tip of the virtual pen and the virtual desktop.

## Claims

### Claim 1

A virtual image display system, comprising:
a virtual image display device; and
an accessory disposed on a hand of a user and coupled to the virtual image display device;
wherein the virtual image display device is configured to:
perform a gesture tracking action of the hand of the user according to positional information of the accessory;
create a virtual pen and a virtual desktop according to gesture tracking information; and
decide whether to generate a virtual handwriting by determining a positional relationship between a pen tip of the virtual pen and the virtual desktop.

### Claim 2 (depends on 1)

The virtual image display system of claim 1, wherein the accessory comprises:
an inertial sensor configured to measure inertial information of the accessory;
a first controller coupled to the inertial sensor; and
a signal sending and receiving interface coupled to the controller,
wherein the signal sending and receiving interface is configured to transmit the inertial information to the virtual image display device.

### Claim 3 (depends on 2)

The virtual image display system of claim 2, wherein the accessory further comprises:
an optical input interface coupled to the first controller, wherein the optical input interface is configured to receive an input command and transmit the input command to the controller.

### Claim 4 (depends on 3)

The virtual image display system of claim 3, wherein the signal sending and receiving interface sends the input command to the virtual image display device, and the virtual image display device displays a function menu according to the input command.

### Claim 5 (depends on 2)

The virtual image display system of claim 2, wherein the virtual image display device comprises:
a display configured to display the virtual pen, the virtual desktop, and the virtual handwriting;
an image capturer configured to capture image information of the hand of the user; and
a second controller coupled to the display, and the second controller is configured to:
receive the inertial information, obtain the positional information of the accessory according to the inertial information and the image information, and thereby perform the gesture tracking action of the hand of the user; and
make the display display the virtual handwriting according to a position of the pen tip of the virtual pen when the pen tip is in contact with the virtual desktop.

### Claim 6 (depends on 4)

The virtual image display system of claim 4, wherein the virtual image display device is configured to, in an initialization action:
perform a pairing action with the accessory;
making the display display the virtual desktop in a virtual space; and
adjust a position of the virtual desktop according to a gesture of the user via the gesture tracking action.

### Claim 7 (depends on 5)

The virtual image display system of claim 5, wherein the virtual image display device is further configured to, in an initialization action:
make the display display the virtual pen in a virtual space;
instruct the user to align a pen tip of a physical pen held with the pen tip of the virtual pen; and
receive an input command transmitted by the user via the accessory to lock a position of the virtual pen.

### Claim 8 (depends on 5)

The virtual image display system of claim 5, wherein the virtual image display device is further configured to, in an initialization action:
make the display display the virtual pen in a virtual space;
make the user hold a pen body of the virtual pen; and
receive an input command transmitted by the user via the accessory to lock a position of the virtual pen.

### Claim 9 (depends on 1)

The virtual image display system of claim 1, wherein the virtual image display device is a head-mounted display.

### Claim 10

A control method of a virtual image display system, comprising:
coupling an accessory disposed on a hand of a user to a virtual image display device;
making the virtual image display device perform a gesture tracking action of the hand of the user according to positional information of the accessory;
making the virtual image display device create a virtual pen and a virtual desktop according to gesture tracking information; and
making the virtual image display device decide whether to generate a virtual handwriting according to a positional relationship between a pen tip of the virtual pen and the virtual desktop.

### Claim 11 (depends on 10)

The control method of claim 10, wherein the step of making the virtual image display device perform the gesture tracking action of the hand of the user according to the positional information of the accessory comprises:
measuring inertial information of the accessory via an inertial sensor;
capturing image information of the hand of the user via an image capturer;
obtaining the positional information of the accessory according to the inertial information and the image information, and thereby performing the gesture tracking action of the hand of the user.

### Claim 12 (depends on 10)

The control method of claim 10, wherein the step of deciding whether to generate the virtual handwriting according to the positional relationship between the pen tip of the virtual pen and the virtual desktop comprises:
making the display display the virtual handwriting according to a position of the pen tip of the virtual pen when the pen tip is in contact with the virtual desktop.

### Claim 13 (depends on 10)

The control method of claim 10, further comprising, in an initialization action:
making the virtual image display device and the accessory perform a pairing action;
making a display of the virtual image display device display the virtual desktop in a virtual space; and
making the virtual image display device adjust a position of the virtual desktop according to a gesture of the user via the gesture tracking action.

### Claim 14 (depends on 13)

The control method of claim 13, further comprising, in the initialization action:
making the display display the virtual pen in the virtual space, wherein an extending direction of the virtual pen is perpendicular to a solid ground;
instructing the user to align a pen tip of a physical pen held with the pen tip of the virtual pen; and
receiving an input command transmitted by the user via the accessory to lock a position of the virtual pen.

### Claim 15 (depends on 13)

The control method of claim 13, further comprising, in the initialization action:
making the display display the virtual pen in the virtual space, wherein an extending direction of the virtual pen is perpendicular to a solid ground;
making the user hold a pen body of the virtual pen; and
receiving an input command transmitted by the user via the accessory to lock a position of the virtual pen.
