# US 12,010,373 B2 · Display apparatus, server apparatus, display system including them, and method for providing content thereof

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,010,373 B2
- Document ID: US12010373B2
- Status: Granted
- Kind: B2
- Application number: 17/159,388
- Filed: 2021-01-27
- Published: 2021-05-20
- Granted: 2024-06-11
- Earliest claimed: 2013-12-27
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Hee-ran Lee; Hee-seob Ryu; Sung-wook Choi; Sung-pil Hwang
- Web page: https://patentgrep.com/patent/US12010373B2

## CPC

- H04N 21/42203: {sound input device, e.g. microphone}
- G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path}
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- H04N 21/431: Generation of visual interfaces {for content selection or interaction}; Content or additional data rendering
- H04N 21/4394: {involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams (arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58)}
- H04N 21/47217: {for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks (specific graphical features in visual interfaces H04N21/4312)}
- H04N 21/4828: {for searching programme descriptors (retrieval of video data G06F16/739)}
- H04N 21/64322: {IP}
- H04N 21/6582: {Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number (arrangements where receivers interact with the broadcast H04H20/38)}
- G10L 2015/223: {Execution procedure of a spoken command}
- H04N 21/4622: {Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet (web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859)}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A display apparatus, including a microphone configured to receive a voice signal and a controller configured to generate a resulting screen corresponding to the voice signal different from a first displayed screen and according to a status of the content of the first displayed screen; to display the resulting screen on the display. Accordingly, the display apparatus provides a different resulting screen according to a status of the display apparatus.

## Claims

### Claim 1

A display apparatus comprising:
a display;
a broadcast receiver;
a communicator;
a voice receiver; and
a processor configured to:
receive a broadcast program from a television broadcast network through the broadcast receiver,
receive a moving image content from an Internet Protocol (IP) network through the communicator,
control the display to display a first plurality of thumbnail images corresponding to a first list of selectable contents associated with the received broadcast program and the received moving image content, wherein at least one first text indicating information of one content of the selectable contents is displayed in one thumbnail image of the first plurality of thumbnail images, and wherein the first plurality of thumbnail images include a thumbnail image associated with the received broadcast program and a thumbnail image associated with the received moving image content,
while the first plurality of thumbnail images corresponding to the first list are displayed on the display, receive a first user voice input via the voice receiver,
based on the first user voice input corresponding to the thumbnail image associated with the received broadcast program, tune a broadcasting channel corresponding to the thumbnail image associated with the received broadcast program and output the received broadcast program received from the broadcasting channel through the broadcast receiver,
based on the first user voice input corresponding to the thumbnail image associated with the received moving image content, receive data corresponding to the received moving image content from the IP network through the communicator and output the received data on a screen of the display,
based on the first user voice input not corresponding to information related to any one of the selectable contents of the first list, control the communicator to transmit data corresponding to the first user voice input, to a server, the information related to any one of the selectable contents of the first list including the at least one first text displayed in the one thumbnail image of the first plurality of thumbnail images, receive, from the server, data corresponding to a search result which is based on the first user voice input, and control the display to display the search result including a second plurality of thumbnail images corresponding to a second list of selectable contents corresponding to the first user voice input,
while the second plurality of thumbnail images corresponding to the second list are displayed on the display, receive a second user voice input via the voice receiver, and
based on the second user voice input corresponding to at least one second text indicating information of one content of the selectable contents of the second list and displayed in one thumbnail image of the second plurality of thumbnail images, control the display to display the one content of the selectable contents of the second list.

### Claim 2 (depends on 1)

The display apparatus of claim 1, wherein the processor is configured to:
based on the first user voice input being received via the voice receiver, control the communicator to transmit the first user voice input to a text converter,
based on the first user voice input being converted into a text by the text converter, control the communicator to receive, from the text converter, the text into which the first user voice input is converted, and
identify whether the first user voice input corresponds to the at least one first text based on the text into which the first user voice input is converted, and
wherein the processor is configured to:
based on the second user voice input being received via the voice receiver, control the communicator to transmit the second user voice input to the text converter,
based on the second user voice input being converted into a text by the text converter, control the communicator to receive, from the text converter, the text into which the second user voice input is converted, and
identify whether the second user voice input corresponds to the at least one second text based on the text into which the second user voice input is converted.

### Claim 3 (depends on 2)

The display apparatus of claim 2, wherein the text converter is included in the server.

### Claim 4 (depends on 1)

The display apparatus of claim 1, wherein the selectable contents of the second list are different from the selectable contents of the first list.

### Claim 5 (depends on 1)

The display apparatus of claim 1, wherein, based on the first user voice input corresponding to the at least one first text, the processor is configured to control the display to display an execution screen of the one content of the selectable contents of the first list.

### Claim 6 (depends on 1)

The display apparatus of claim 1, wherein, based on the second user voice input corresponding to the at least one second text, the processor is configured to control the display to display an execution screen of the one content of the selectable contents of the second list.

### Claim 7 (depends on 1)

The display apparatus of claim 1, wherein the first list comprises at least one of title information, alias title information and identification information of the selectable contents of the first list, and
wherein the second list comprises at least one of title information, alias title information and identification information of the selectable contents of the second list.

### Claim 8 (depends on 1)

The display apparatus of claim 1, wherein the one content of the selectable contents of the first list is a content of the broadcast program, and the processor is configured to control the broadcast receiver to receive the one content of the selectable contents of the first list to be displayed on the display.

### Claim 9 (depends on 1)

The display apparatus of claim 1, wherein the processor is further configured to, based on the second user voice input corresponding to the at least one second text, control the display to display, on a first location of the display, first information on a first content that is related to the at least one second text and has been already viewed and display, on a second location of the display, second information on a second content that is related to the at least one second text and is newly viewable, the first location and the second location being different from each other.

### Claim 10

A method for providing content information of a display apparatus including a display, the method comprising:
receiving a broadcast program from a television broadcast network;
receiving a moving image content from an Internet Protocol (IP) network;
displaying a first plurality of thumbnail images corresponding to a first list of selectable contents associated with the received broadcast program and the received moving image content, wherein at least one first text indicating information of one content of the selectable contents is displayed in one thumbnail image of the first plurality of thumbnail images and wherein the first plurality of thumbnail images include a thumbnail image associated with the received broadcast program and a thumbnail image associated with the received moving image content;
while the first plurality of thumbnail images corresponding to the first list are displayed on the display, receiving a first user voice input;
based on the first user voice input corresponding to the thumbnail image associated with the received broadcast program, tuning a broadcasting channel corresponding to the thumbnail image associated with the received broadcast program and outputting the received broadcast program received from the broadcasting channel through a broadcast receiver of the display apparatus;
based on the first user voice input corresponding to the thumbnail image associated with the received moving image content, receiving data corresponding to the received moving image content from the IP network through a communicator of the display apparatus and outputting the received data on a screen of the display;
based on the first user voice input not corresponding to information related to any one of the selectable contents of the first list, transmitting data corresponding to the first user voice input, to a server, the information related to any one of the selectable contents of the first list including the at least one first text displayed in the one thumbnail image of the first plurality of thumbnail images, receiving, from the server, data corresponding to a search result which is based on the first user voice input, displaying the search result including a second plurality of thumbnail images corresponding to a second list of selectable contents corresponding to the first user voice input;
while the second plurality of thumbnail images corresponding to the second list are displayed on the display, receiving a second user voice input; and
based on the second user voice input corresponding to at least one second text indicating information of one content of the selectable contents of the second list and displayed in one thumbnail image of the second plurality of thumbnail images, displaying the one content of the selectable contents of the second list.

### Claim 11 (depends on 10)

The method of claim 10, further comprising:
transmitting the first user voice input to a text converter;
based on the first user voice input being converted into a text by the text converter, receiving, from the text converter, the text into which the first user voice input is converted;
identifying whether the first user voice input corresponds to the at least one first text based on the text into which the first user voice input is converted;
transmitting the second user voice input to the text converter;
based on the second user voice input being converted into a text by the text converter, receiving, from the text converter, the text into which the second user voice input is converted; and
identifying whether the second user voice input corresponds to the at least one second text based on the text into which the second user voice input is converted.

### Claim 12 (depends on 11)

The method of claim 11, wherein the text converter is included in the server.

### Claim 13 (depends on 10)

The method of claim 10, wherein the selectable contents of the second list are different from the selectable contents of the first list.

### Claim 14 (depends on 10)

The method of claim 10, wherein the displaying the one content of the selectable contents of the first list comprises, based on the first user voice input corresponding to the at least one first text, displaying an execution screen of the one content of the selectable contents of the first list.

### Claim 15 (depends on 10)

The method of claim 10, wherein the displaying the one content of the selectable contents of the second list comprises, based on the second user voice input corresponding to the at least one second text, displaying an execution screen of the one content of the selectable contents of the second list.

### Claim 16 (depends on 10)

The method of claim 10, wherein the first list comprises at least one of title information, alias title information and identification information of the selectable contents of the first list, and
wherein the second list comprises at least one of title information, alias title information and identification information of the selectable contents of the second list.

### Claim 17 (depends on 10)

The method of claim 10, wherein the one content of the selectable contents of the first list is a content of the broadcast program.

### Claim 18 (depends on 10)

The method of claim 10, wherein the displaying of the one content of the selectable contents of the second list comprises, based on the second user voice input corresponding to the at least one second text, displaying, on a first location of the display first, information on a first content that is related to the at least one second text and has been already viewed and displaying, on a second location of the display, second information on a second content that is related to the at least one second text and is newly viewable, the first location and the second location being different from each other.