# US 2018/0322870 A1 · Performing tasks and returning audio and visual feedbacks based on voice command

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0322870 A1
- Document ID: US20180322870A1
- Status: Published application
- Kind: A1
- Application number: 15/872,966
- Filed: 2018-01-16
- Published: 2018-11-08
- Earliest claimed: 2017-05-08
- Assignees: Kt Corporation
- Applicants: Kt Corporation
- Inventors: Sang-Wook LEE; Hyun-Jung KANG; Sun-Jong KWON; Mi-Hee KIM; Jin-Han KIM; Gyu-Tae BAEK; Joo-Won SUNG; Kyeong-Jin OH; Jin-Young OH; Ji-Hui IM; Du-Seong CHANG; Won-Seock CHOI
- Web page: https://patentgrep.com/patent/US20180322870A1

## CPC

- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
- H04N 21/42203: {sound input device, e.g. microphone}
- H04N 21/4394: {involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams (arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58)}
- G10L 2015/223: {Execution procedure of a spoken command}
- G10L 2015/227: {of the speaker; Human-factor methodology}
- G10L 2015/228: {of application context}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An artificial intelligence voice interactive system may provide various services to a user in response to a voice command. The system may perform at least one of: receiving user speech from a user; transmitting the received user speech to the central server; receiving at least one of control signals and audio and video answers, which are generated based on an analysis and interpretation result of the user speech, from at least one of the central server, the internal service servers, and the external service servers; and outputting the received audio and video answers through at least one of speakers and a display coupled to the user interactive device and controlling at least one device according to the control signals

## Claims

### Claim 1

A method for providing voice interactive services to a user through an artificial intelligence voice interactive system including a user interactive device, a central server, a plurality of internal service servers, and a plurality of external service servers each connected to the central server through a communication network, the method of the user interactive device comprising:
receiving user speech from a user;
transmitting the received user speech to the central server;
receiving at least one of control signals and audio and video answers, which are generated based on an analysis and interpretation result of the user speech, from at least one of the central server, the internal service servers, and the external service servers; and
outputting the received audio and video answers through at least one of speakers and a display coupled to the user interactive device and controlling at least one device according to the control signals.

### Claim 2 (depends on 1)

The method of claim 1, comprising:
displaying video contents on the display coupled to the user interactive device; and
outputting the received audio and video answers while displaying the video contents.

### Claim 3 (depends on 2)

The method of claim 2, comprising:
controlling the displaying of the video contents according to the control signals.

### Claim 4 (depends on 2)

The method of claim 2, comprising:
displaying video data of the audio and video feedback on at least one part of the display in an overlapping manner while displaying the video contents on the display.

### Claim 5 (depends on 4)

The method of claim 4, comprising:
displaying the audio and video feedback in a form of text on at least one part of the display in an overlapping manner while displaying the video contents of the display.

### Claim 6 (depends on 2)

The method of claim 2, comprising:
receiving user speech from a user while displaying the video contents on the display; and
controlling the displaying of the video contents according to control signals and audio and video feedbacks, generated based on analysis and interpretation results of the received user speech.

### Claim 7 (depends on 2)

The method of claim 2, comprising:
outputting, through a first speaker coupled to the display, a part of audio data of the video contents being displayed on the display; and
outputting audio data of the audio and video feedback through a second speaker of the user interactive device.

### Claim 8 (depends on 2)

The method of claim 2, comprising:
outputting i) a part of audio data of the video contents being displayed on the display and ii) audio data of the audio and video feedback simultaneously through a second speaker of the user interactive device.

### Claim 9

A server for providing a service based on user speech, the server comprising:
a communication circuit configured to communication with other entities including a user interactive device and external service servers;
a memory configured to store data and information;
a processor configured to:
receive user speech from a user interactive device through the communication circuit and extract voice commands from the user speech;
analyze and interpret the voice commands based on context information of the user interactive device; and
generate audio and video feedback as an answer to the user speech and transmit the audio and video feedback to the user interactive device.

### Claim 10 (depends on 9)

The server of claim 9, wherein the processor is configured to: determine intention of the user speech by deducing omitted information in the user speech based on user preference information.

### Claim 11 (depends on 9)

The server of claim 9, wherein:
the communication circuit is configured to receive information on a voice command history of the user interactive device; and
the processor is configured to determine intention of the user speech based on the voice command history of the user interactive device.

### Claim 12 (depends on 9)

The server of claim 9, wherein:
the communication circuit is configured to receive information on a frequency of each service performed by the user interactive device; and
the processor is configured to determine intention of the user speech by controlling and assigning a weight to each service based on the frequency thereof.

### Claim 13 (depends on 9)

The server of claim 9, wherein:
the communication circuit is configured to receive information on a history of previously voice commands received by the user interactive device; and
the processor is configured to generate a different pattern audio and video feedback when the user interactive device receives the same voice commands repeatedly.

### Claim 14 (depends on 9)

The server of claim 9, wherein:
the communication circuit is configured to receive information on a usage duration of the user interactive device; and
the processor is configured to generate audio and video feedback based on a system familiarity level determined based on the usage duration.

### Claim 15 (depends on 9)

The server of claim 9, wherein:
the communication circuit is configured to receive information on context of the user interactive device; and
the processor is configured to interpret and analyze the user speech and the voice commands based on the information on the context of the user interactive device.

### Claim 16

A server for providing a service based on user speech, the server comprising:
a communication circuit configured to communication with other entities including a user interactive device and external service servers;
a memory configured to store data and information;
a processor configured to:
receive user speech from a user interactive device through the communication circuit;
determine whether a service determination is required by interpreting the user speech;
when the service determination is required, analyze the user speech based on context information of the user interactive device, generate audio and video feedback as an answer to the user speech, and transmit the audio and video feedback to the user interactive device; and
when the service determination is not required, transmit the user speech interpreting result to the user interactive device.

### Claim 17 (depends on 16)

The server of claim 16, wherein the processor is configured to determine whether a service determined corresponding to the user speech requires an external service server.

### Claim 18 (depends on 17)

The server of claim 17, wherein the processor is configured to request the external service server to perform at least one task corresponding to the determined service, generate at least one control signal and audio and video feedback based on the result of performing the task.
