# US 10,455,269 B2 · Systems and methods for automated extraction of closed captions in real time or near real-time and tagging of streaming data for advertisements > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,455,269 B2 - Document ID: US10455269B2 - Status: Granted - Kind: B2 - Application number: 16/170,944 - Filed: 2018-10-25 - Published: 2019-02-28 - Granted: 2019-10-22 - Earliest claimed: 2009-12-18 - Assignees: Crossbar Media Group, Inc - Applicants: Crossbar Media Group, Inc - Inventors: Samir Abed - Web page: https://patentgrep.com/patent/US10455269B2 ## CPC - H04N 21/2668: Creating a channel for a dedicated end-user group, e.g. insertion of targeted commercials based on end-user profiles {(information retrieval from the Internet by querying with filtering and personalisation G06F16/9535; arrangements for replacing or switching information during the broadcast H04H20/10; push services over packet-switching network H04L12/1859; adaptation of message content in packet-switching networks H04L51/063)} - G06F 16/7844: {using original textual content or text extracted from visual content or transcript of audio data} - G06F 16/951: Indexing; Web crawling techniques - H04H 60/375: {Commercial} - H04H 60/48: for recognising items expressed in broadcast information - H04H 60/66: for using the result on distributors' side - H04N 7/0882: {for the transmission of character code signals, e.g. for teletext (circuits for the digital non-picture data signal H04N7/035)} - H04N 21/2181: {comprising remotely distributed storage units, e.g. when movies are replicated over a plurality of video servers (distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097)} - H04N 21/233: Processing of audio elementary streams {(monitoring, identification or recognition of audio in broadcast systems H04H60/58)} - H04N 21/23418: {involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 21/2353: {specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata} - H04N 21/25883: {being end-user demographical data, e.g. age, family status or address (arrangements for identifying locations of users in broadcast systems H04H60/52)} - H04N 21/25891: {being end-user preferences (retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18)} - H04N 21/4332: {by placing content in organized collections, e.g. local EPG data repository (interfaces, Database management systems or updating for information retrieval G06F16/23; details of retrieval of video data and associated meta data in video database G06F16/739)} - H04N 21/4334: {Recording operations (recording of a television signal H04N5/76; arrangements for recording or accumulating broadcast information or broadcast-related information H04H60/27)} - H04N 21/4348: {Demultiplexing of additional data and video streams} - H04N 21/4394: {involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams (arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58)} - H04N 21/4398: {involving reformatting operations of audio signals (details of audio signal transcoding G10L19/173)} - H04N 21/440236: {by media transcoding, e.g. video is transformed into a slideshow of still pictures, audio is converted into text} - H04N 21/4828: {for searching programme descriptors (retrieval of video data G06F16/739)} - H04N 21/4882: {for displaying messages, e.g. warnings, reminders (arrangements for providing short real-time information to substation in data switching networks H04L12/1895)} - H04N 21/4888: {for displaying teletext characters} - H04N 21/812: {involving advertisement data (advertising per se G06Q30/02)} - H04N 21/8352: involving content or source identification data, e.g. Unique Material Identifier [UMID] - H04N 21/8456: {by decomposing the content in the time domain, e.g. in time segments} - H04N 21/858: Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04H: BROADCAST COMMUNICATION (multiplex communication H04J; pictorial communication aspects of broadcast systems H04N) ## Abstract System and methods for finding and analyzing targeted content from audio and video content sources, including means and methods for extracting captions from audio and video content sources; searching the captions for a mention of at least one target; extracting audio and video segments relating to the at least one target; delivering extracted audio and video segments to a user device; harvesting social media data relevant to the at least one target; analyzing the search results in correlation with the social media data for target content. ## Claims ### Claim 1 A system for targeted content analysis, comprising: at least one device and a server platform constructed and configured for network communication; wherein the at least one device is operable to receive a live broadcast and/or stream audio and video content, and extract captions of the live broadcast and/or the audio and video content in real time; wherein the server platform is operable to receive extracted captions from the at least one device, search the extracted captions for at least one keyword relating to targeted content, thereby creating search result data, harvest social media data relevant to the targeted content for a predetermined period of time from the Internet, and calculate an impact of the targeted content by correlating the search result data with the social media data; and wherein the server platform is a peer-to-peer platform. ### Claim 2 (depends on 1) The system of claim 1, wherein the server platform is cloud-based. ### Claim 3 (depends on 1) The system of claim 1, wherein the peer-to-peer platform is operable to record the extracted captions on a blockchain. ### Claim 4 (depends on 1) The system of claim 1, wherein the server platform is further operable to extract audio and video segments relevant to the targeted content from the audio and video content, and transmit the audio and video segments to the at least one device. ### Claim 5 (depends on 4) The system of claim 4, wherein the at least one device is further operable to share the audio and video segments on at least one sharing platform including iTunes, Twitter, Facebook, and/or other social media or digital media platforms. ### Claim 6 (depends on 5) The system of claim 5, wherein the server platform is further operable to receive feedback and/or a rating of the audio and video segments shared on the at least one sharing platform, and analyze the targeted content based on the feedback and/or the rating in correlation with the search result data and the social media data. ### Claim 7 (depends on 1) The system of claim 1, wherein the server platform is further operable to identify at least one social media tag relevant to the at least one keyword for the targeted content, wherein the at least one social media tag comprises at least one hashtag relevant to the at least one keyword for the targeted content. ### Claim 8 (depends on 1) The system of claim 1, wherein the server platform is operable to harvest the social media data in a predetermined geographic area. ### Claim 9 (depends on 1) The system of claim 1, wherein the server platform is operable to harvest the social media data from at least one social media site or at least one social media application including Twitter, Facebook, Instagram, Snapchat, and/or other web-based sites or applications for groups. ### Claim 10 (depends on 1) The system of claim 1, wherein the server platform is further operable to determine the impact of the targeted content in real time or near real time. ### Claim 11 (depends on 1) The system of claim 1, wherein the at least one device comprises a graphical user interface operable to display and facilitate comparisons between the search result data and the social media data. ### Claim 12 (depends on 1) The system of claim 1, wherein the server platform is operable to authorize and authenticate the at least one device. ### Claim 13 A system for targeted content analysis, comprising: at least one device and a computing platform constructed and configured for network communication; wherein the at least one device is operable to extract captions of audio or video; wherein the computing platform is operable to receive extracted captions from the at least one device, search the extracted captions for a targeted content based on user profile and preferences, thereby creating search result data, harvest social media data relevant to the targeted content for a predetermined time period, and calculate an impact of the targeted content based on the social media data correlated in time with the search result data; and wherein the computing platform is a distributed and decentralized platform on a blockchain, and wherein the extracted captions are recorded on the blockchain. ### Claim 14 (depends on 13) The system of claim 13, wherein the computing platform is cloud-based. ### Claim 15 (depends on 13) The system of claim 13, wherein the computing platform is further operable to extract audio or video segments relevant to the targeted content, deliver at least one alert regarding extracted audio or video segments to the at least one device, and format the extracted captions to a more human readable text in free-form format. ### Claim 16 A system for determining the effectiveness of an advertisement campaign, comprising: at least one device and a computing platform constructed and configured for network communication; wherein the computing platform is a blockchain platform; wherein the at least one device is configured to extract captions of at least one audio or video related to the advertisement campaign; and wherein the computing platform is configured to receive extracted captions from the at least one device, search the extracted captions for at least one keyword related to the advertisement campaign, thereby creating search result data, monitoring social media activities related to the advertisement campaign, thereby creating social media data, and determine an effectiveness of the advertisement campaign based on the social media data correlated with the search result data. ### Claim 17 (depends on 16) The system of claim 16, wherein the computing platform is further operable to generate at least one advertisement tag code (ATC) for the advertisement campaign, mark the at least one audio or video related to the advertisement campaign with the at least one ATC, and extract audio or video segments from the at least one audio or video related to the advertisement campaign.