# US 2021/0241771 A1 · Electronic device and method for controlling the electronic device thereof

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0241771 A1
- Document ID: US20210241771A1
- Status: Published application
- Kind: A1
- Application number: 17/248,616
- Filed: 2021-01-29
- Published: 2021-08-05
- Earliest claimed: 2020-01-31
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Jeonghyun YUN
- Web page: https://patentgrep.com/patent/US20210241771A1

## CPC

- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path}
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06F 40/279: Recognition of textual entities
- G06F 40/30: Semantic analysis
- G06N 3/04: Architecture, e.g. interconnection topology
- G10L 15/16: using artificial neural networks
- G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning}
- G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
- G06F 40/295: Named entity recognition
- G06N 3/08: Learning methods
- G10L 15/32: Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems
- G10L 2015/088: {Word spotting}
- G10L 2015/223: {Execution procedure of a spoken command}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

An electronic device and a method for controlling the electronic device are disclosed. The method includes receiving a trigger speech of a user; entering a speech recognition mode to recognize a speech command of the user in response, and transmitting information to enter the speech recognition mode to at least one external device located at home. Further, the method includes obtaining a first speech information corresponding to a speech uttered by a user from a microphone included in the electronic device and receiving at least one second speech information corresponding to a speech uttered by the user from the at least one external device; identifying a task corresponding to a speech uttered by the user and an external device to perform the task based on the first speech information and the at least one second speech information; and transmitting, to the identified external device, information about the task.

## Claims

### Claim 1

A method of controlling an electronic device, the method comprising:
receiving a trigger speech of a user;
entering a speech recognition mode to recognize a speech command of the user in response to the trigger speech, and transmitting information to enter the speech recognition mode to at least one external device located at home;
obtaining a first speech information corresponding to a speech uttered by a user from a microphone included in the electronic device while operating in the speech recognition mode and receiving at least one second speech information corresponding to a speech uttered by the user from the at least one external device;
identifying a task corresponding to a speech uttered by the user and an external device to perform the task based on the first speech information and the at least one second speech information; and
transmitting, to the external device, information about the task.

### Claim 2 (depends on 1)

The method of claim 1, wherein the identifying further comprises:
obtaining first text information corresponding to the first speech information and at least one second text information corresponding to at least one second speech information by inputting the first speech information and at least one second speech information to an automatic speech recognition (ASR) module; and
obtaining final text information corresponding to a speech uttered by the user based on the first text information and the at least one second text information.

### Claim 3 (depends on 2)

The method of claim 2, wherein the obtaining the final text information comprises:
based on the first text information and the second text information being identical, identifying the first text information as the final text information, and
based on the first text information and the second text information being different, obtaining final text information by inputting the first text information and the second text information to an artificial neural network model.

### Claim 4 (depends on 2)

The method of claim 2, further comprising:
obtaining information about a task corresponding to a speech uttered by the user by inputting the final text information to a natural language understanding (NLU) module.

### Claim 5 (depends on 2)

The method of claim 2, wherein the final text information comprises at least one keyword, and
wherein the identifying further comprises identifying the external device to perform a task corresponding to the speech uttered by the user based on the at least one keyword.

### Claim 6 (depends on 3)

The method of claim 3, wherein the final text information comprises a plurality of keywords corresponding to each of a syllable of the speech uttered by the user, and
wherein the identifying further comprises identifying the external device to perform a task corresponding to the speech uttered by the user based on the plurality of keywords.

### Claim 7 (depends on 5)

The method of claim 5, wherein the identifying the external device comprises identifying the external device to perform a task through a weight assigned to each of at least one keyword included in the final text information.

### Claim 8 (depends on 1)

The method of claim 1, wherein the transmitting to the at least one external device comprises further transmitting information to cause a second electronic device, which is reproducing an image or audio, among the at least one external device receiving the information, to enter a mute mode.

### Claim 9 (depends on 1)

The method of claim 1, wherein speech recognition engines included in the electronic device and the at least one external device are identical.

### Claim 10

An electronic device, comprising:
a communication interface;
an input interface;
a memory storing at least one instruction; and
a processor configured to control the electronic device by executing the at least one instruction stored in the memory,
wherein the processor is further configured to:
receive a trigger speech of a user through the input interface,
enter a speech recognition mode to recognize a user's speech command in response to the trigger speech, and transmit information to enter a speech recognition mode to at least one external device located at home through the communication interface,
obtain a first speech information corresponding to a speech uttered by a user from a microphone included in the input interface while operating in a speech recognition mode and receive at least one second speech information corresponding to a speech uttered by the user from the at least one external device through the communication interface,
identify a task corresponding to a speech uttered by the user and an external device to perform the task based on the first speech information and the at least one second speech information; and
transmit, to the external device, information about the task through the communication interface.

### Claim 11 (depends on 10)

The electronic device of claim 10, wherein the processor is further configured to:
obtain first text information corresponding to the first speech information and at least one second text information corresponding to at least one second speech information by inputting the first speech information and at least one second speech information to an automatic speech recognition (ASR) module, and
obtain final text information corresponding to a speech uttered by the user based on the first text information and the at least one second text information.

### Claim 12 (depends on 11)

The electronic device of claim 11, wherein the processor is further configured to:
based on the first text information and the second text information being identical, identify the first text information as the final text information, and
based on the first text information and the second text information being different, obtain final text information by inputting the first text information and the second text information to an artificial neural network model.

### Claim 13 (depends on 11)

The electronic device of claim 11, wherein the processor is further configured to obtain information about a task corresponding to a speech uttered by the user by inputting the final text information to a natural language understanding (NLU) module.

### Claim 14 (depends on 11)

The electronic device of claim 11, wherein the final text information comprises at least one keyword, and
wherein the processor is further configured to identify the external device to perform a task corresponding to the speech uttered by the user based on the at least one keyword.

### Claim 15 (depends on 11)

The electronic device of claim 11, wherein the final text information comprises a plurality of keywords corresponding to each of a syllable of the speech uttered by the user, and
wherein the processor is further configured to identify the external device to perform a task corresponding to the speech uttered by the user based on the plurality of keywords.

### Claim 16 (depends on 14)

The electronic device of claim 14, wherein the processor is further configured to identify the external device to perform a task through a weight assigned to each of at least one keyword included in the final text information.

### Claim 17 (depends on 10)

The electronic device of claim 10, wherein the processor is further configured to further transmit information to cause a second electronic device which is reproducing an image or audio, among the at least one external device receiving the information, to enter a mute mode.

### Claim 18 (depends on 10)

The electronic device of claim 10, wherein speech recognition engines included in the electronic device and the at least one external device are identical.

### Claim 19

A non-transitory computer readable medium comprising instructions, that when executed by a processor of an electronic device, cause the electronic device to:
receive a trigger speech of a user through the input interface,
enter a speech recognition mode to recognize a user's speech command in response to the trigger speech, and transmit information to enter a speech recognition mode to at least one external device located at home through the communication interface,
obtain a first speech information corresponding to a speech uttered by a user from a microphone included in the input interface while operating in a speech recognition mode and receive at least one second speech information corresponding to a speech uttered by the user from the at least one external device through the communication interface,
identify a task corresponding to a speech uttered by the user and an external device to perform the task based on the first speech information and the at least one second speech information; and
transmit, to the external device, information about the task through the communication interface.

### Claim 20 (depends on 19)

The computer readable medium of claim 19, wherein the instructions obtain first text information corresponding to the first speech information and at least one second text information corresponding to at least one second speech information by inputting the first speech information and at least one second speech information to an automatic speech recognition (ASR) module, and
obtain final text information corresponding to a speech uttered by the user based on the first text information and the at least one second text information.
