# US 10,373,613 B2 · Dual-mode voice control method, device, and user terminal

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,373,613 B2
- Document ID: US10373613B2
- Status: Granted
- Kind: B2
- Application number: 15/363,990
- Filed: 2016-11-29
- Published: 2017-06-08
- Granted: 2019-08-06
- Earliest claimed: 2015-12-08
- Assignees: Guangzhou Shenma Mobile Information Technology Co., Ltd.
- Applicants: Guangzhou Shenma Mobile Information Technology Co., Ltd.
- Inventors: Yajun Wang; Tuwenchang Si; Na Wang; Yi Peng; Sishou Zheng; Xiaoli Fu; Chao Li; Wei Kang; Yining Chen; Zejun Ma
- Web page: https://patentgrep.com/patent/US10373613B2

## CPC

- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G06F 3/04817: using icons (graphical or visual programming using iconic symbols G06F8/34)
- G06F 3/04883: for inputting data by handwriting, e.g. gesture or text
- G10L 15/32: Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems
- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- G06F 3/04847: Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G10L 15/05: Word boundary detection
- G10L 2015/223: {Execution procedure of a spoken command}
- G10L 2015/228: {of application context}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A dual-mode voice control method is disclosed. The method may comprise determining whether a user has executed an operation of activating an operate-to-speak stop determination mode in a voice input interface. The method may further comprise, in response to determining that the user has executed the operation of activating the operate-to-speak stop determination mode, determining whether a microphone is in a busy state. The method may further comprise, in response to determining that the microphone is in the busy state, switching a voice mode from a directly-speak automatic stop determination mode to the operate-to-speak stop determination mode. Before the user executes the operation of activating the operate-to-speak stop determination mode, the voice mode is in the directly-speak automatic stop determination mode if the microphone is in the busy state.

## Claims

### Claim 1

A dual-mode voice control method, comprising:
determining whether a user has executed an operation of activating a manual speak stop determination mode in a voice input interface, wherein during the manual speak stop determination mode, stop of voice input is determined based on a user operation of ending the manual speak stop determination mode, regardless of whether the user carries out voice input at a microphone;
determining that the user has executed the operation of activating the manual speak stop determination mode while the microphone is in a busy state, wherein the voice input interface is in an automatic speak stop determination when the microphone is in the busy state, wherein during the automatic speak stop determination mode, stop of voice input is determined based on no voice input for a certain length of time; and
in response to determining that the user has executed the operation of activating the manual speak stop determination mode while the microphone is in the busy state, switching a voice mode of the voice input interface from the automatic speak stop determination mode to the manual speak stop determination mode such that stop of voice input is determined based on the user operation of ending the manual speak stop determination mode instead of based on no voice input for the certain length of time.

### Claim 2 (depends on 1)

The method of claim 1, wherein determining whether the user has executed the operation of activating the manual speak stop determination mode in the voice input interface comprises:
determining whether the user has executed a start operation of activating the manual speak stop determination mode in the voice input interface;
in response to detecting the start operation, monitoring the time length during which the user executes the start operation; and
in response to determining that the time length reaches a preset threshold, determining that the operation of activating the manual speak stop determination mode in the voice input interface is detected.

### Claim 3 (depends on 1)

The method of claim 1, wherein determining whether the user has executed the operation of activating the manual speak stop determination mode in the voice input interface comprises:
determining whether the user has executed a start operation of activating the manual stop determination mode in the voice input interface;
in response to determining the start operation, determining whether the microphone is in an idle state; and
in response to determining that the microphone is in the idle state, setting the automatic speak stop determination mode as the voice mode, monitoring the time length during which the user executes the start operation, and in response to determining that the time length reaches a preset threshold, determining that the operation of activating the manual speak stop determination mode in the voice input interface is detected.

### Claim 4 (depends on 3)

The method of claim 3, further comprising:
acquiring voice information expressed by the user in the voice input interface;
monitoring an operation of ending the manual speak stop determination mode by the user in the voice input interface;
in response to determining that the user has executed the operation of ending the manual speak stop determination mode, determining whether the automatic speak stop determination mode or the manual speak stop determination mode is currently active;
in response to determining that the automatic speak stop determination mode is currently active, continuing acquiring the voice information expressed by the user; and
in response to determining that the manual speak stop determination mode is currently active, stopping acquiring the voice information expressed by the user.

### Claim 5

A dual-mode voice control method, comprising:
determining whether a user has executed a start operation of an operation of activating a manual speak stop determination mode in a voice input interface, wherein during the manual speak stop determination mode, stop of voice input is determined based on a user operation of ending the manual speak stop determination mode, regardless of whether the user carries out voice input at a microphone;
determining that the user has executed the start operation of activating the manual speak stop determination mode while the microphone is in an idle state;
in response to determining that the user has executed the start operation of activating the manual speak stop determination mode while the microphone is in the idle state, setting an automatic speak stop determination mode as the voice mode of the voice input interface, and monitoring the time length during which the user executes the start operation, wherein during the automatic speak stop determination mode, stop of voice input is determined based on no voice input for a certain length of time; and
in response to determining that the time length reaches a preset threshold, determining that the operation of activating the manual speak stop determination mode in the voice input interface is detected, and switching the voice mode of the voice input interface from the automatic speak stop determination mode to the manual speak stop determination mode such that stop of voice input is determined based on the user operation of ending the manual speak stop determination mode instead of based on no voice input for the certain length of time.

### Claim 6 (depends on 5)

The method of claim 5, further comprising:
acquiring voice information expressed by the user in the voice input interface;
monitoring an operation of ending the manual speak stop determination mode by the user in the voice input interface;
in response to determining that the user has executed the operation of ending the manual speak stop determination mode, determining whether the automatic speak stop determination mode or the manual speak stop determination mode is currently active;
in response to determining that the automatic speak stop determination mode is currently active, continuing acquiring the voice information expressed by the user; and
in response to determining that the manual speak stop determination mode is currently active, stopping acquiring the voice information expressed by the user.

### Claim 7

A dual-mode voice control device operable in a user terminal, the device comprising:
a processor; and
a non-transitory computer-readable memory storing instructions that, when executed by the processor, cause the processor to:
monitor whether a user has executed an operation of activating a manual speak stop determination mode in a voice input interface, wherein during the manual speak stop determination mode, stop of voice input is determined based on a user operation of ending the manual speak stop determination mode, regardless of whether the user carries out voice input at a microphone; and
determining that the user has executed the operation of activating the manual speak stop determination mode while the microphone is in a busy state, wherein the voice input interface is in an automatic speak stop determination when the microphone is in the busy state, wherein during the automatic speak stop determination mode, stop of voice input is determined based on no voice input for a certain length of time; and
in response to determining that that the user has executed the operation of activating the manual speak stop determination mode while the microphone is in the busy state, switching a voice mode of the voice input interface from the automatic speak stop determination mode to the manual speak stop determination mode such that stop of voice input is determined based on the user operation of ending the manual speak stop determination mode instead of based on no voice input for the certain length of time.

### Claim 8 (depends on 7)

The device of claim 7, wherein, to monitor whether the user has executed the operation of activating the manual speak stop determination mode in the voice input interface, the instructions, when executed by the processor, are configured to cause the processor to:
determine whether the user has executed a start operation of activating the manual speak stop determination mode in the voice input interface; and
in response to determining the start operation, monitor the time length during which the user executes the start operation, and when the time length reaches a preset threshold, determine that the operation of activating the manual speak stop determination mode in the voice input interface is detected.

### Claim 9 (depends on 7)

The device of claim 7, wherein, to monitor whether the user has executed the operation of activating the manual speak stop determination mode in the voice input interface, the instructions, when executed by the processor, are configured to cause the processor to:
determine whether the user has executed a start operation of activating the manual speak stop determination mode in the voice input interface;
in response to determining the start operation, determine whether a microphone is in an idle state; and
in response to determining the microphone is in the idle state, set the automatic speak stop determination mode as the voice mode, monitor the time length during which the user executes the start operation, and when the time length reaches a preset threshold, determine that the operation of activating the manual speak stop determination mode in the voice input interface is detected.

### Claim 10 (depends on 9)

The device of claim 9, wherein the instructions, when executed by the processor, are further configured to cause the processor to:
acquire voice information expressed by the user in the voice input interface;
monitor an operation of ending the manual speak stop determination mode by the user in the voice input interface;
in response to determining that the user has executed the operation of ending the manual speak stop determination mode, determine whether the automatic speak stop determination mode or the manual speak stop determination mode is currently active;
in response to determining that the automatic speak stop determination mode is currently active, continue acquiring the voice information expressed by the user; and
in response to determining that the manual speak stop determination mode is currently active, stop acquiring the voice information expressed by the user.

### Claim 11

A dual-mode voice control device operable in a user terminal, the device comprising:
a processor; and
a non-transitory computer-readable memory storing instructions that, when executed by the processor, cause the processor to:
determine whether a user has executed a start operation of activating a manual speak stop determination mode in a voice input interface, wherein during the manual speak stop determination mode, stop of voice input is determined based on a user operation of ending the manual speak stop determination mode, regardless of whether the user carries out voice input at a microphone;
determine that the user has executed the start operation of activating the manual speak stop determination mode while the microphone is in an idle state;
in response to determining that the user has executed the start operation of activating the manual speak stop determination mode while the microphone is in the idle state:
set an automatic speak stop determination mode as the voice mode of the voice input interface, wherein during the automatic speak stop determination mode, stop of voice input is determined based on no voice input for a certain length of time,
monitor the time length during which the user executes the start operation, and
when the time length reaches a preset threshold, determine that the operation of activating the manual speak stop determination mode in the voice input interface is detected, and switch the voice mode of the voice input interface from the automatic speak stop determination mode to the manual speak stop determination mode such that stop of voice input is determined based on the user operation of ending the manual speak stop determination mode instead of based on no voice input for the certain length of time.

### Claim 12 (depends on 11)

The device of claim 11, wherein the instructions, when executed by the processor, are further configured to cause the processor to:
acquire voice information expressed by the user in the voice input interface;
monitor an operation of ending the manual speak stop determination mode by the user in the voice input interface;
in response to determining that the user has executed the operation of ending the manual speak stop determination mode, determine whether the automatic speak stop determination mode or the manual speak stop determination mode is currently active;
in response to determining that the automatic speak stop determination mode is currently active, continue acquiring the voice information expressed by the user; and
in response to determining that the manual speak stop determination mode is currently active, stop acquiring the voice information expressed by the user.

### Claim 13

A user terminal, comprising:
a microphone, a touch screen, a non-transitory computer-readable memory, and a processor, wherein:
the memory stores instructions that, when executed by the processor, cause the user terminal to:
monitor whether a user has executed an operation of activating a manual speak stop determination mode in a voice input interface, wherein during the manual speak stop determination mode, stop of voice input is determined based on a user operation of ending the manual speak stop determination mode, regardless of whether the user carries out voice input at the microphone; and
determining that the user has executed the operation of activating the manual speak stop determination mode while the microphone is in a busy state, wherein the voice input interface is in an automatic speak stop determination when the microphone is in the busy state, wherein during the automatic speak stop determination mode, stop of voice input is determined based on no voice input for a certain length of time; and
in response to determining that that the user has executed the operation of activating the manual speak stop determination mode while the microphone is in the busy state, switching a voice mode of the voice input interface from the automatic speak stop determination mode to the manual speak stop determination mode such that stop of voice input is determined based on the user operation of ending the manual speak stop determination mode instead of based on no voice input for the certain length of time.

### Claim 14 (depends on 13)

The user terminal of claim 13, wherein, to monitor whether the user has executed the operation of activating the manual speak stop determination mode in the voice input interface, the memory stores instructions that, when executed by the processor, cause the user terminal to:
determine whether the user has executed a start operation of activating the manual speak stop determination mode in the voice input interface; and
in response to determining the start operation, monitor the time length during which the user executes the start operation, and when the time length reaches a preset threshold, determine that the operation of activating the manual peak stop determination mode in the voice input interface is detected.

### Claim 15 (depends on 13)

The user terminal of claim 13, wherein, to monitor whether the user has executed the operation of activating the manual speak stop determination mode in the voice input interface, the memory stores instructions that, when executed by the processor, cause the user terminal to:
determine whether the user has executed a start operation of activating the manual speak stop determination mode in the voice input interface; and
in response to determining the start operation, determine whether a microphone is in an idle state,
in response to determining the microphone is in the idle state, set the automatic speak stop determination mode as the voice mode, monitor the time length during which the user executes the start operation, and when the time length reaches a preset threshold, determine that the operation of activating the manual speak stop determination mode in the voice input interface is detected.

### Claim 16 (depends on 13)

The user terminal of claim 13, wherein the memory stores instructions that, when executed by the processor, cause the user terminal to:
acquire voice information expressed by the user in the voice input interface;
monitor an operation of ending the manual speak stop determination mode by the user in the voice input interface;
in response to determining that the user has executed the operation of ending the manual speak stop determination mode, determine whether the automatic speak stop determination mode or the manual stop determination mode is currently active;
in response to determining that the automatic speak stop determination mode is currently active, continue acquiring the voice information expressed by the user; and
in response to determining that the manual stop determination mode is currently active, stop acquiring the voice information expressed by the user.

### Claim 17

A user terminal, comprising:
a microphone, a touch screen, a non-transitory computer-readable memory, and a processor, wherein:
the memory stores instructions that, when executed by the processor, cause the user terminal to:
determine whether a user has executed a start operation of activating a manual speak stop determination mode in a voice input interface, wherein during the manual speak stop determination mode, stop of voice input is determined based on a user operation of ending the manual speak stop determination mode, regardless of whether the user carries out voice input at the microphone;
determine that the user has executed the start operation of activating the manual speak stop determination mode while the microphone is in an idle state;
in response to determining that the user has executed the start operation of activating the manual speak stop determination mode while the microphone is in the idle state:
set an automatic speak stop determination mode as the voice mode of the voice input interface, wherein during the automatic speak stop determination mode, stop of voice input is determined based on no voice input for a certain length of time,
monitor the time length during which the user executes the start operation, and
when the time length reaches a preset threshold, determine that the operation of activating the manual speak stop determination mode in the voice input interface is detected, and switch the voice mode of the voice input interface from the automatic speak stop determination mode to the manual speak stop determination mode such that stop of voice input is determined based on the user operation of ending the manual speak stop determination mode instead of based on no voice input for the certain length of time.

### Claim 18 (depends on 17)

The user terminal of claim 17, wherein the memory stores instructions that, when executed by the processor, further cause the user terminal to:
acquire voice information expressed by the user in the voice input interface;
monitor an operation of ending the manual speak stop determination mode by the user in the voice input interface; and
in response to determining that the user has executed the operation of ending the manual speak stop determination mode, determine whether the automatic speak stop determination mode or the manual speak stop determination mode is currently active; in response to determining that the automatic speak stop determination mode is currently active, continue acquiring the voice information expressed by the user; and in response to determining that the manual speak stop determination mode is currently active, stop acquiring the voice information expressed by the user.