# US 2024/0004477 A1 · Keyboard perspective method and apparatus for virtual reality device, and virtual reality device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0004477 A1
- Document ID: US20240004477A1
- Status: Published application
- Kind: A1
- Application number: 18/037,969
- Filed: 2021-11-12
- Published: 2024-01-04
- Earliest claimed: 2020-11-23
- Assignees: Qingdao Pico Technology Co., Ltd.
- Applicants: Qingdao Pico Technology Co., Ltd
- Inventors: Ming ZHANG
- Web page: https://patentgrep.com/patent/US20240004477A1

## CPC

- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06V 20/20: in augmented reality scenes
- G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

The present application discloses a keyboard perspective method and apparatus for a virtual reality device, and the virtual reality device. The method comprises: recognizing hand actions of two hands of a user; if the hand actions of the two hands of the user meet a preset activation action, activating a keyboard perspective function of a virtual reality device; recognizing hand positions of the two hands of the user under the keyboard perspective function; and determining a keyboard perspective display area according to the hand positions of the two hands of the user so as to display a physical keyboard in a real scene in the keyboard perspective display area. Compared with traditional keyboard perspective schemes, use experience of the user is greatly improved.

## Claims

### Claim 1

A keyboard see-through method for a virtual reality device, comprising:
recognizing hand actions of two hands of a user;
activating a keyboard see-through function of the virtual reality device if the hand actions of the two hands of the user meet a preset activation action;
determining hand positions of the two hands of the user under the keyboard see-through function; and
determining a keyboard see-through display area according to the hand positions of the two hands of the user so as to display a physical keyboard of a real world in the keyboard see-through display area.

### Claim 2 (depends on 1)

The method according to claim 1, wherein activating the keyboard see-through function of the virtual reality device if the hand actions of the two hands of the user meet the preset activation action comprises:
determining that the hand actions of the two hands of the user meet the preset activation action if the hand actions of the two hands of the user are actions in which the two hands extend downward and approach.

### Claim 3 (depends on 1)

The method according to claim 1, wherein determining the keyboard see-through display area according to the hand positions of the two hands of the user comprises:
determining a circumscribed rectangular area of the two hands according to the hand positions of the two hands of the user; and
enlarging the determined circumscribed rectangular area by a preset multiple, and taking the enlarged circumscribed rectangular area as the keyboard see-through display area.

### Claim 4 (depends on 1)

The method according to claim 1, wherein determining the keyboard see-through display area according to the hand positions of the two hands of the user comprises:
determining a circumscribed square area of any palm of the user;
determining a ratio length and a ratio width for expansion according to the size of the circumscribed square area; and
determining a center position of a left palm and a center position of a right palm of the user, connecting the center position of the left palm with the center position of the right palm to form a connecting line, and expanding the ratio length to a vertical direction and expanding the ratio width to a horizontal direction according to a midpoint of the connecting line to obtain the keyboard see-through display area.

### Claim 5 (depends on 1)

The method according to claim 1, further comprising:
turning off the keyboard see-through function of the virtual reality device if the hand actions of the two hands of the user meet a preset turn-off action.

### Claim 6

(canceled)

### Claim 7

(canceled)

### Claim 8

(canceled)

### Claim 9

(canceled)

### Claim 10

A virtual reality device, comprising: a processor and a memory for storing computer-executable instructions,
wherein the computer-executable instructions, when executed by the processor, cause the processor to implement to operations comprising:
recognizing hand actions of two hands of a user;
activating a keyboard see-through function of the virtual reality device if the hand actions of the two hands of the user meet a preset activation action;
determining hand positions of the two hands of the user under the keyboard see-through function; and
determining a keyboard see-through display area according to the hand positions of the two hands of the user so as to display a physical keyboard of a real world in the keyboard see-through display area.

### Claim 11 (depends on 10)

The virtual reality device according to claim 10, wherein activating the keyboard see-through function of the virtual reality device if the hand actions of the two hands of the user meet the preset activation action comprises:
determining that the hand actions of the two hands of the user meet the preset activation action if the hand actions of the two hands of the user are actions in which the two hands extend downward and approach.

### Claim 12 (depends on 10)

The virtual reality device according to claim 10, wherein determining the keyboard see-through display area according to the hand positions of the two hands of the user comprises:
determining a circumscribed rectangular area of the two hands according to the hand positions of the two hands of the user; and
enlarging the determined circumscribed rectangular area by a preset multiple, and taking the enlarged circumscribed rectangular area as the keyboard see-through display area.

### Claim 13 (depends on 10)

The virtual reality device according to claim 10, wherein determining the keyboard see-through display area according to the hand positions of the two hands of the user comprises:
determining a circumscribed square area of any palm of the user;
determining a ratio length and a ratio width for expansion according to the size of the circumscribed square area; and
determining a center position of a left palm and a center position of a right palm of the user, connecting the center position of the left palm with the center position of the right palm to form a connecting line, and expanding the ratio length to a vertical direction and expanding the ratio width to a horizontal direction according to a midpoint of the connecting line to obtain the keyboard see-through display area.

### Claim 14

A non-transitory computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program, when executed by a processor, cause the processor to perform operations comprising:
recognizing hand actions of two hands of a user;
activating a keyboard see-through function of the virtual reality device if the hand actions of the two hands of the user meet a preset activation action;
determining hand positions of the two hands of the user under the keyboard see-through function; and
determining a keyboard see-through display area according to the hand positions of the two hands of the user so as to display a physical keyboard of a real world in the keyboard see-through display area.

### Claim 15 (depends on 14)

The non-transitory computer-readable storage medium according to claim 14, wherein activating the keyboard see-through function of the virtual reality device if the hand actions of the two hands of the user meet the preset activation action comprises:
determining that the hand actions of the two hands of the user meet the preset activation action if the hand actions of the two hands of the user are actions in which the two hands extend downward and approach.

### Claim 16 (depends on 14)

The non-transitory computer-readable storage medium according to claim 14, wherein determining the keyboard see-through display area according to the hand positions of the two hands of the user comprises:
determining a circumscribed rectangular area of the two hands according to the hand positions of the two hands of the user; and
enlarging the determined circumscribed rectangular area by a preset multiple, and taking the enlarged circumscribed rectangular area as the keyboard see-through display area.

### Claim 17 (depends on 14)

The non-transitory computer-readable storage medium according to claim 14, wherein determining the keyboard see-through display area according to the hand positions of the two hands of the user comprises:
determining a circumscribed square area of any palm of the user;
determining a ratio length and a ratio width for expansion according to the size of the circumscribed square area; and
determining a center position of a left palm and a center position of a right palm of the user, connecting the center position of the left palm with the center position of the right palm to form a connecting line, and expanding the ratio length to a vertical direction and expanding the ratio width to a horizontal direction according to a midpoint of the connecting line to obtain the keyboard see-through display area.
