# US 12,190,882 B2 · Method and apparatus for generating general voice commands and augmented reality display

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,190,882 B2
- Document ID: US12190882B2
- Status: Granted
- Kind: B2
- Application number: 18/254,258
- Filed: 2021-11-30
- Published: 2024-01-18
- Granted: 2025-01-07
- Earliest claimed: 2020-12-01
- Assignees: Hangzhou Lingban Technology Co. Ltd.
- Applicants: Hangzhou Lingban Technology Co. Ltd.
- Inventors: Weiming Liu
- Web page: https://patentgrep.com/patent/US12190882B2

## CPC

- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 15/06: Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice (G10L15/14 takes precedence)
- G10L 15/08: Speech classification or search
- G10L 2015/223: {Execution procedure of a spoken command}
- G10L 2015/228: {of application context}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

The present disclosure relates to a method and an apparatus for generating general voice commands, and the method includes: obtaining View tree content of a display interface of an application; traversing information nodes in the View tree content, and configuring different voice commands for different information nodes based on attributes of the information nodes; and aggregating all voice commands in the display interface, and mixing and filtering the commands to form a final voice command set.

## Claims

### Claim 1

A method for generating general voice commands, wherein the method comprises:
obtaining View tree content of a display interface of an application executed on an augmented reality device;
traversing information nodes in the View tree content, and configuring different voice commands for different information nodes based on attributes of the information nodes; and
aggregating all voice commands in the display interface, and mixing and filtering the commands to form a final voice command set,
wherein the aggregating all voice commands in the display interface, and mixing and filtering the commands to form a final voice command set comprises: deduplicating duplicate voice commands by retaining a first traversed voice command and filtering out a later traversed voice command,
wherein the aggregating all voice commands in the display interface, and mixing and filtering the commands to form a final voice command set further comprises:
comparing the final voice command set with a voice command set that is set at a previous time, and if the final voice command set remains unchanged, considering current setting as invalid setting; or
otherwise, sending the final voice command set to a voice assistant executed on the augmented reality device to complete setting of the voice command set, wherein the voice assistant is further configured to receive a voice command via a microphone of the augmented reality device and recognize the voice command based on the final voice command set.

### Claim 2 (depends on 1)

The method for generating general voice commands according to claim 1, wherein the View tree content of the display interface of the application is obtained by using an accessibility service function of an Android system.

### Claim 3 (depends on 1)

The method for generating general voice commands according to claim 1, wherein the traversing information nodes in the View tree content, and configuring different voice commands for different information nodes based on attributes of the information nodes comprises:
recursively traversing the information nodes in the View tree content level by level and node by node, and in a traversing process, selecting each policy selector for each information node; and
storing, in each policy selector, a command generated after the selection, and configuring a corresponding voice command type.

### Claim 4 (depends on 3)

The method for generating general voice commands according to claim 3, wherein the policy selector comprises a tap policy selector, a text policy selector, a list policy selector, a remote control policy selector, and a self-defined protocol policy selector.

### Claim 5 (depends on 4)

The method for generating general voice commands according to claim 4, wherein the View tree content is sequentially traversed in a sequence of the tap policy selector, the text policy selector, the list policy selector, the remote control policy selector, and the self-defined protocol policy selector.

### Claim 6 (depends on 1)

The method for generating general voice commands according to claim 1, further comprising: performing a specific function according to the voice command issued by a user.

### Claim 7 (depends on 6)

The method for generating general voice commands according to claim 6, wherein the performing a specific function according to the voice command issued by a user comprises:
sending a voice command prompt to the user, performing hit determining on the voice command issued by the user, and performing the specific function based on a determining result.

### Claim 8 (depends on 7)

The method for generating general voice commands according to claim 7, wherein the sending a voice command prompt to the user comprises:
adding a lightweight system floating layer at a top layer of a display view of a device, and sending a command prompt based on a command type of each voice command.