# US 10,319,382 B2 · Multi-level voice menu

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,319,382 B2
- Document ID: US10319382B2
- Status: Granted
- Kind: B2
- Application number: 16/001,769
- Filed: 2018-06-06
- Published: 2018-10-04
- Granted: 2019-06-11
- Earliest claimed: 2013-01-30
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Michael J. LeBeau; Clifford Ivar Nass
- Web page: https://patentgrep.com/patent/US10319382B2

## CPC

- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G02B 27/017: {Head mounted}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G10L 21/10: Transforming into visible information
- G10L 25/48: specially adapted for particular use
- H04R 1/08: Mouthpieces; {Microphones;} Attachments therefor
- G02B 2027/014: {comprising information/image processing systems}
- G02B 2027/0178: {Eyeglass type (eyeglass details G02C)}
- G10L 2015/223: {Execution procedure of a spoken command}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K)

## Abstract

Methods, apparatus, and computer-readable media are described herein related to a user interface (UI) that can be implemented on a head-mountable device (HMD). The UI can include a voice-navigable UI. The voice-navigable UI can include a voice navigable menu that includes one or more menu items. The voice-navigable UI can also present a first visible menu that includes at least a portion of the voice navigable menu. In response to a first utterance comprising one of the one or more menu items, the voice-navigable UI can modify the first visible menu to display one or more commands associated with the first menu item. In response to a second utterance comprising a first command, the voice-navigable UI can invoke the first command. In some embodiments, the voice-navigable UI can display a second visible menu, where the first command can be displayed above other menu items in the second visible menu.

## Claims

### Claim 1

A method, comprising:
receiving, by a computing device, a first utterance comprising a hotword;
in response to receiving the first utterance comprising the hotword, displaying, by the computing device, a first visible menu comprising one or more commands associated with the hotword, wherein each displayed command comprises a text version of a corresponding speech command from a plurality of speech commands;
receiving, by the computing device, a second utterance comprising a speech command corresponding to a first command from the one or more commands; and
in response to receiving the second utterance, invoking the first command;
receiving, by the computing device, a third utterance comprising the speech command corresponding to a second command from the one or more commands, the second command differing from the first command; and
after receiving the third utterance:
invoking the second command; and
displaying at least a portion of a third visible menu, wherein the third visible menu includes the second command and at least one menu item of the one or more menu items, and wherein the third visible menu displays the second command above the at least one menu item.

### Claim 2 (depends on 1)

The method of claim 1,
wherein the third visible menu further includes the first command; and
wherein an order in which the first command and the second command are displayed in the third visible menu is determined based on at least a frequency of use for the first command and a frequency of use for the second command.

### Claim 3

A method, comprising:
receiving, by a computing device, a first utterance comprising a hotword, wherein the first utterance comprising the hotword is received in a first context of the voice navigable interface;
in response to receiving the first utterance comprising the hotword, displaying, by the computing device, a first visible menu comprising one or more commands associated with the hotword, wherein each displayed command comprises a text version of a corresponding speech command from a plurality of speech commands, and wherein the first visible menu is displayed in a second context of the voice navigable interface;
receiving, by the computing device, a second utterance comprising a speech command corresponding to a first command from the one or more commands; and
in response to receiving the second utterance, invoking the first command;
subsequently, in the first context of the voice navigable interface, receiving a third utterance comprising a second hotword,
in response to receiving the third utterance comprising the second hotword, displaying a second visible menu comprising one or more second commands associated with the second hotword, wherein each displayed second command comprises a text version of a corresponding speech command from a second plurality of speech commands;
receiving, by the computing device, a fourth utterance comprising a speech command corresponding to a command from the second plurality of speech commands; and
in response to receiving the fourth utterance, invoking the corresponding command from the second plurality of speech commands.

### Claim 4

A method, comprising:
receiving, by a computing device, an utterance comprising a hotword;
in response to receiving the utterance comprising the hotword, displaying, by the computing device, a first visible menu comprising at least a portion of a voice navigable menu, wherein the voice navigable menu includes one or more menu items, and wherein each menu item comprises a text version of a corresponding speech command from a plurality of speech commands;
receiving, by the computing device, a first utterance comprising a speech command corresponding to a first menu item from the one or more menu items;
in response to receiving the first utterance, modifying the first visible menu to display one or more commands associated with the first menu item, wherein each displayed command comprises a text version of a corresponding speech command from the plurality of speech commands;
receiving, by the computing device, a second utterance comprising a speech command corresponding to a first command from the one or more commands; and
in response to receiving the second utterance, invoking the first command;
displaying at least a portion of a second visible menu, wherein the second visible menu includes the first command and at least one menu item of the one or more menu items, wherein each menu item comprises a text version of a corresponding speech command from a plurality of speech commands, and wherein the second visible menu displays the first command above the at least one menu item;
receiving a third utterance at the computing device, the third utterance comprising the speech command corresponding to a second command from the one or more commands, the second command differing from the first command;
after receiving the third utterance, invoking the second command; and
displaying at least a portion of a third visible menu, wherein the third visible menu includes the second command and at least one menu item of the one or more menu items, and wherein the third visible menu displays the second command above the at least one menu item.

### Claim 5 (depends on 4)

The method of claim 4, further comprising:
initially, displaying a graphic representation corresponding to the hotword.

### Claim 6 (depends on 4)

The method of claim 4,
wherein the third visible menu further includes the first command; and
wherein an order in which the first command and the second command are displayed in the third visible menu is determined based on at least a frequency of use for the first command and a frequency of use for the second command.

### Claim 7 (depends on 6)

The method of claim 6,
wherein the third visible menu further includes the first command; and
wherein an order in which the first command and the second command are displayed in the third visible menu is determined based on at least a recency of use for the first command and a recency of use for the second command.

### Claim 8 (depends on 6)

The method of claim 6,
wherein the third visible menu further includes the first command; and
wherein an order in which the first command and the second command are displayed in the third visible menu is determined based on at least a combination of a frequency of use and a recency of use for the first command and a combination of a frequency of use and a recency of use for the second command.

### Claim 9 (depends on 4)

The method of claim 4, wherein the first menu item comprises an identification of a category.

### Claim 10 (depends on 4)

The method of claim 4, wherein at least one menu item of the one or more menu items comprises an identification of a category.

### Claim 11 (depends on 4)

The method of claim 4, wherein modifying the first visible menu to display the one or more commands associated with the first menu item comprises expanding the first visible menu to display the one or more commands in-line with the first menu item.

### Claim 12 (depends on 4)

The method of claim 4, wherein modifying the first visible menu to display the one or more commands associated with the first menu item comprises displaying the one or more commands as a submenu.

### Claim 13

A computing device comprising:
a processor; and
a non-transitory computer-readable medium configured to store at least program instructions that, when executed by the processor, cause the computing device to carry out functions comprising:
displaying a first visible menu comprising at least a portion of a voice navigable menu, wherein the voice navigable menu includes one or more menu items, and wherein each menu item comprises a text version of a corresponding speech command from a plurality of speech commands,
receiving a first utterance, the first utterance comprising the speech command corresponding to a first menu item from the one or more menu items,
in response to receiving the first utterance, modifying the first visible menu to display one or more commands associated with the first menu item, wherein each command comprises a text version of a corresponding speech command from the plurality of speech commands,
receiving a second utterance at the computing device, the second utterance comprising the speech command corresponding to a first command from the one or more commands,
in response to receiving the second utterance, invoking the first command;
displaying at least a portion of a second visible menu, wherein the second visible menu includes the first command and at least one menu item of the one or more menu items, and wherein the second visible menu displays the first command above the at least one menu item;
receiving a third utterance, the third utterance comprising a second command, the second command differing from the first command;
after receiving the third utterance, invoking the second command; and
displaying at least a portion of a third visible menu, wherein the third visible menu includes the second command and at least one menu item of the one or more menu items, and wherein the third visible menu displays the second command above the at least one menu item.

### Claim 14 (depends on 13)

The computing device of claim 13, wherein:
the third visible menu further includes the first command; and
an order in which the first command and the second command are displayed in the third visible menu is determined based on at least a frequency of use for the first command and a frequency of use for the second command.