# US 10,375,440 B2 · Display device, server, and method of controlling the display device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,375,440 B2 - Document ID: US10375440B2 - Status: Granted - Kind: B2 - Application number: 15/632,552 - Filed: 2017-06-26 - Published: 2017-10-12 - Granted: 2019-08-06 - Earliest claimed: 2014-03-05 - Assignees: Samsung Electronics Co., Ltd. - Applicants: Samsung Electronics Co., Ltd. - Inventors: Jae-seok Jang; Tae-ung Jung - Web page: https://patentgrep.com/patent/US10375440B2 ## CPC - H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - G06F 16/51: Indexing; Data structures therefor; Storage structures - G06F 18/22: Matching criteria, e.g. proximity measures - G06V 20/46: {Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames} - G06V 40/1365: {Matching; Classification} - H04N 5/50: Tuning indicators; Automatic tuning control - H04N 21/4348: {Demultiplexing of additional data and video streams} - H04N 21/435: Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream - H04N 21/4394: {involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams (arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58)} - H04N 21/4722: for requesting additional data associated with the content - H04N 21/4758: {for providing answers, e.g. voting} - H04N 21/6547: comprising parameters, e.g. for client setup - H04N 21/6582: {Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number (arrangements where receivers interact with the broadcast H04H20/38)} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A display device is provided. The display device includes an outputter configured to output a content including a plurality of content parts, a signal processor configured to extract a first fingerprint corresponding to a first content part among the plurality of content parts of the content from at least one of a video signal and an audio signal constituting the content while the content is output, a communicator configured to transmit the first fingerprint to a server, and receive a second fingerprint corresponding to a second content part among the plurality of content parts of the content and first event information corresponding to the second fingerprint from the server, a storage configured to store the second fingerprint and the first event information, and a controller configured to control the outputter to output the first event information in coordination with the second content part of the content being output. ## Claims ### Claim 1 A display device comprising: a signal receiver configured to receive content signal comprising at least one of a video signal; an outputter configured to output an image based on the content signal; a communicator configured to communicate with a server; and a controller configured: to control the communicator to transmit, to the server, first identification information for identifying the image output by the outputter, to receive, from the server second identification information and event information which correspond to the first identification information, and to perform a process corresponding to the event information based on a third identification information extracted from the content signal matching the second identification information. ### Claim 2 (depends on 1) The display device as claimed in claim 1, wherein the performing the process comprises controlling the outputter to output an image corresponding to the event information once it is determined that the third identification information matches the second identification information. ### Claim 3 (depends on 1) The display device as claimed in claim 1, wherein the controller is further configured to obtain the first identification information from the video signal or from an electronic program guide (EPG) information. ### Claim 4 (depends on 3) The display device as claimed in claim 3, wherein the first identification information is obtained from the EPG information rather than from the video signal if the EPG information is included in the content signal. ### Claim 5 (depends on 1) The display device as claimed in claim 1, wherein at least one of the first identification information or the second identification information includes at least one selected from among a video frame, edge information of an object included in the video frame, caption information included in the video frame, logo information, frequency information of a specific period in an audio signal, and audio signal peak information over the specific period in the audio signal. ### Claim 6 (depends on 1) The display device as claimed in claim 1, wherein the event information includes at least one selected from among shopping information, channel recommendation information, broadcasting schedule information, and comment collection information. ### Claim 7 (depends on 1) The display device as claimed in claim 1, wherein the first identification information, the second identification information and the third identification information are video fingerprints. ### Claim 8 (depends on 1) The display device as claimed in claim 1, wherein the communicator is further configured receive a fourth identification information and event information associated with the fourth identification information from the server, and wherein if a fifth identification information extracted from the content signal matches the fourth identification information received from the server, the controller is further configured to perform a process corresponding to the event information associated with the fourth identification information. ### Claim 9 A server comprising: a storage; a communicator configured to communicate with a display device; and a controller configured to: store a plurality of fingerprints in the storage, in response to a first fingerprint being received from the display device, to identify a content associated with the first fingerprint based on a fingerprint among the plurality of fingerprints matching the first fingerprint, and control the communicator to transmit event information related to the identified content and a second fingerprint corresponding to the event information to the display device. ### Claim 10 (depends on 9) The server as claimed in claim 9, wherein at least one of the first fingerprint or the second fingerprint includes at least one selected from among a video frame, edge information of an object included in the video frame, caption information included in the video frame, logo information, frequency information of a specific period in an audio signal and audio signal peak information over the specific period in the audio signal. ### Claim 11 (depends on 9) The server as claimed in claim 9, wherein the event information includes at least one selected from among shopping information, channel recommendation information, broadcasting schedule information, and comment collection information. ### Claim 12 A method of providing event information of a display device, the method comprising: receiving content signal comprising a video signal; outputting an image based on the content signal; transmitting, to a server, first identification information for identifying the image output by an outputter; receiving, from the server, second identification information and event information which correspond to the first identification information; and performing a process corresponding to the event information based on whether a third identification information extracted from the content signal matching the second identification information. ### Claim 13 (depends on 12) The method as claimed in claim 12, wherein the performing a process comprises outputting an image corresponding to the event information once it is determined that the third identification information matches the second identification information. ### Claim 14 (depends on 12) The method as claimed in claim 12, wherein the method further comprises obtaining the first identification information from the video signal or from an electronic program guide (EPG) information. ### Claim 15 (depends on 14) The method as claimed in claim 14, wherein the first identification information is obtained from the EPG information rather than from the video signal if the EPG information is included in the content signal. ### Claim 16 (depends on 12) The method as claimed in claim 12, wherein at least one of the first identification information or the second identification information includes at least one selected from among a video frame, edge information of an object included in the video frame, caption information included in the video frame, logo information, frequency information of a specific period in an audio signal, and audio signal peak information over the specific period in the audio signal. ### Claim 17 (depends on 12) The method as claimed in claim 12, wherein the event information includes at least one selected from among shopping information, channel recommendation information, broadcasting schedule information, and comment collection information. ### Claim 18 (depends on 12) The method as claimed in claim 12, wherein the first identification information, the second identification information and the third identification information are video fingerprints. ### Claim 19 (depends on 12) The method as claimed in claim 12, further comprising: receiving a fourth identification information and event information associated with the fourth identification information from the server; and if a fifth identification information extracted from the content signal matches the fourth identification information performing a process corresponding the event information associated with the fourth identification information. ### Claim 20 A method of providing event information of a server, the method comprising: storing a plurality of fingerprints in a storage of the server; in response to a first fingerprint being received from a display device through a communicator of the server, identifying a content associated with the first fingerprint based on a fingerprint among the plurality of fingerprints matching the first fingerprint, and controlling the communicator to transmit event information related to the identified content and a second fingerprint corresponding to the event information to the display device through the communicator. ### Claim 21 (depends on 20) The method as claimed in claim 20, wherein at least one of the first fingerprint or the second fingerprint includes at least one selected from among a video frame, edge information of an object included in the video frame, caption information included in the video frame, logo information, frequency information of a specific period in an audio signal and audio signal peak information over the specific period in the audio signal. ### Claim 22 (depends on 20) The method as claimed in claim 20, wherein the event information includes at least one selected from among shopping information, channel recommendation information, broadcasting schedule information, and comment collection information.