# US 9,003,440 B2 · Method and system for synchronization of messages to content utilizing automatic content recognition > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,003,440 B2 - Document ID: US9003440B2 - Status: Granted - Kind: B2 - Application number: 13/730,644 - Filed: 2012-12-28 - Published: 2013-08-08 - Granted: 2015-04-07 - Earliest claimed: 2012-02-07 - Assignees: Turner Broadcasting System, Inc. - Applicants: Turner Broadcasting System, Inc. - Inventors: Nishith Kumar Sinha; Donald Jude Loheide; Matthew Paul Giles - Web page: https://patentgrep.com/patent/US9003440B2 ## CPC - H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range - G06F 16/24558: {Binary matching operations} - G06T 1/0021: {Image watermarking} - H04N 21/00: Selective content distribution, e.g. interactive television or video on demand [VOD] (real-time bi-directional transmission of motion video data H04N7/14 {; broadcast or conference over packet switching networks H04L12/18}) - H04N 21/251: {Learning process for intelligent management, e.g. learning user preferences for recommending movies (details of learning user preferences for the retrieval of video data in a video database G06F16/739; computer systems using learning methods G06N3/08)} - H04N 21/25866: {Management of end-user data (customer care in data networks H04L41/5077)} - H04N 21/4147: PVR [Personal Video Recorder] (H04N5/76 takes precedence {; arrangements for broadcast with accumulation-type receivers H04H20/40}) - H04N 21/4223: Cameras (H04N23/00 takes precedence) - H04N 21/432: Content retrieval operation from a local storage medium, e.g. hard-disk {(details of retrieval of video data and associated meta data in video databases G06F16/739)} - H04N 21/435: Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream - H04N 21/441: Acquiring end-user identification {, e.g. using personal code sent by the remote control or by inserting a card} - H04N 21/442: Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk {(arrangements for monitoring broadcast services or broadcast-related services H04H60/29; arrangements for identifying or recognising characteristics with a direct linkage to broadcast information H04H60/35; monitoring of user activities for profile generation for accessing a video database G06F16/739; monitoring in wireless networks H04W24/00)} - H04N 21/4532: {involving end-user characteristics, e.g. viewer profile, preferences (monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18)} - H04N 21/4667: {Processing of monitored end-user data, e.g. trend analysis based on the log file of viewer selections} - H04N 21/4722: for requesting additional data associated with the content - H04N 21/478: Supplemental services, e.g. displaying phone caller identification, shopping application - H04N 21/4784: receiving rewards - H04N 21/812: {involving advertisement data (advertising per se G06Q30/02)} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract An automatic content recognition (ACR) enabled (ACR-enabled) connected TV device may be operable to present, utilizing ACR, an overlay starting at a time that is synchronized to an identified event in presented content. The overlay may comprise one or more messages associated with an online social network. The presented content may comprise a program, a movie, a game or a commercial. The one or more messages may be pre-generated, before presentation of the content, for presentation synchronized to the identified event during the presentation of the content. The one or more messages may be generate during broadcast or streaming of a program or a movie, and the presented content, which comprises the program or the movie, may be presented during a time-delayed playback. The ACR-enabled connected TV device may track, utilizing the ACR, how each of the presented one or more messages is actually being viewed and/or interacted with. ## Claims ### Claim 1 A method, comprising: in an automatic content recognition (ACR) enabled (ACR-enabled) connected TV device, wherein: said ACR-enabled connected TV device comprises an ACR module that communicates with an ACR system that is external to said ACR-enabled connected TV device, and said ACR system supports different video fingerprint technologies from multiple vendors; said ACR module providing fingerprints to said ACR system for matching; said ACR module receiving information and/or content from said ACR system based on said matching; said ACR system assigns same interactive event identifiers to different sets of video and audio fingerprints that are generated by said different video and audio fingerprint technologies from said multiple vendors based on timing of said assignment of said interactive event identifiers; and said interactive event identifiers are utilized in said ACR-enabled connected TV device to trigger events: presenting, utilizing ACR, an overlay starting at a time synchronized to an identified event in presented content, wherein said presented overlay comprises one or more messages associated with an online social network. ### Claim 2 (depends on 1) The method according to claim 1, wherein said one or more messages are: pre-generated and prerecorded prior to said content being presented; and presented, during said presentation of said content, at a time that is synchronized to said identified event. ### Claim 3 (depends on 1) The method according to claim 1, comprising: when said one or more messages are being presented, presenting, utilizing said ACR, one or more additional messages associated with said online social network in said presented overlay, wherein said one or more additional messages are generated during said presentation of said content. ### Claim 4 (depends on 1) The method according to claim 1, wherein: said one or more messages are generated during broadcast or streaming of a program or a movie; and said presented content, which comprises said program or said movie, is presented during a time-delayed playback. ### Claim 5 (depends on 1) The method according to claim 1, wherein: said presented content comprises a program or a movie; and said one or more messages are generated by a person associated with said program or said movie. ### Claim 6 (depends on 5) The method according to claim 5, comprising presenting, utilizing said ACR, another overlay during presentation of an identified advertisement in a commercial break, wherein: said another overlay comprises one or more other messages associated with said online social network; and said one or more other messages are also generated by said person. ### Claim 7 (depends on 1) The method according to claim 1, wherein said presented content comprises a program, a movie, a game or a commercial associated with a network. ### Claim 8 (depends on 1) The method according to claim 1, wherein, when said ACR-enabled connected TV device is paired with a paired device utilizing a companion application running on said paired device, said overlay is also presented on said paired device. ### Claim 9 (depends on 1) The method according to claim 1, comprising tracking, utilizing said ACR, how each of said presented one or more messages is actually being viewed and/or interacted with. ### Claim 10 (depends on 1) The method according to claim 1, comprising: when an outgoing message associated with said online social network is generated and displayed during presentation of said content, identifying, utilizing said ACR, a particular event in said presented content, wherein said particular event occurs at a time synchronized to a beginning of presentation of said outgoing message. ### Claim 11 A system, comprising: an automatic content recognition (ACR) enabled (ACR-enabled) connected TV device, wherein: said ACR-enabled connected TV device comprises an ACR module that communicates with an ACR system that is external to said ACR-enabled connected TV device, and said ACR system supports different video fingerprint technologies from multiple vendors; said ACR module providing fingerprints to said ACR system for matching; said ACR module receiving information and/or content from said ACR system based on said matching; said ACR system assigns same interactive event identifiers to different sets of video and audio fingerprints that are generated by said different video and audio fingerprint technologies from said multiple vendors based on timing of said assignment of said interactive event identifiers; and said interactive event identifiers are utilized in said ACR-enabled connected TV device to trigger events, said ACR-enabled connected TV device being operable to: present, utilizing ACR, an overlay starting at a time synchronized to an identified event in presented content, wherein said presented overlay comprises one or more messages associated with an online social network. ### Claim 12 (depends on 11) The system according to claim 11, wherein said one or more messages are: pre-generated and prerecorded prior to said content being presented; and presented, during said presentation of said content, at a time that is synchronized to said identified event. ### Claim 13 (depends on 11) The system according to claim 11, wherein, when said one or more messages are being presented, said ACR-enabled connected TV device is operable to: present, utilizing said ACR, one or more additional messages associated with said online social network in said presented overlay, and said one or more additional messages are generated during said presentation of said content. ### Claim 14 (depends on 11) The system according to claim 11, wherein: said one or more messages are generated during broadcast or streaming of a program or a movie; and said presented content, which comprises said program or said movie, is presented during a time-delayed playback. ### Claim 15 (depends on 11) The system according to claim 11, wherein: said presented content comprises a program or a movie; and said one or more messages are generated by a person associated with said program or said movie. ### Claim 16 (depends on 15) The system according to claim 15, wherein: said ACR-enabled connected TV device is operable to present, utilizing said ACR, another overlay during presentation of an identified advertisement in a commercial break, wherein: said another overlay comprises one or more other messages associated with said online social network; and said one or more other messages are also generated by said person. ### Claim 17 (depends on 11) The system according to claim 11, wherein said presented content comprises a program, a movie, a game or a commercial associated with a network. ### Claim 18 (depends on 11) The system according to claim 11, wherein, when said ACR-enabled connected TV device is paired with a paired device utilizing a companion application running on said paired device, said overlay is also presented on said paired device. ### Claim 19 (depends on 11) The system according to claim 11, wherein: said ACR-enabled connected TV device is operable to track, utilizing said ACR, how each of said presented one or more messages is actually being viewed and/or interacted with; and when an outgoing message associated with said online social network is generated and displayed during presentation of said content, said ACR-enabled connected TV device is operable to identify, utilizing said ACR, a particular event in said presented content, and said particular event occurs at a time synchronized to a beginning of presentation of said outgoing message. ### Claim 20 A system, comprising: an automatic content recognition (ACR) enabled (ACR-enabled) connected TV device, wherein: said ACR-enabled connected TV device comprises an ACR module that communicates with an ACR system that is external to said ACR-enabled connected TV device, and said ACR system comprises an abstraction layer that supports different video fingerprint technologies from multiple vendors; said ACR module providing fingerprints to said ACR system for matching; said ACR module receiving information and/or content from said ACR system based on said matching; said abstraction layer enables said ACR system to assign same interactive event identifiers to different sets of video and audio fingerprints that are generated by said different video and audio fingerprint technologies from said multiple vendors based on timing of said assignment of said interactive event identifiers; and said interactive event identifiers are utilized in said ACR-enabled connected TV device to trigger events, said ACR-enabled connected TV device being operable to: present, utilizing ACR, an overlay starting at a time synchronized to an identified event in presented content, wherein said presented overlay comprises one or more messages associated with an online social network.