# US 11,908,191 B2 · System and method for merging asynchronous data sources > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,908,191 B2 - Document ID: US11908191B2 - Status: Granted - Kind: B2 - Application number: 17/449,697 - Filed: 2021-10-01 - Published: 2022-04-07 - Granted: 2024-02-20 - Earliest claimed: 2020-10-01 - Assignees: Stats LLC - Applicants: Stats LLC - Inventors: Alex Ottenwess; Matthew Scott; Ken Rhodes; Patrick Joseph Lucey - Web page: https://patentgrep.com/patent/US11908191B2 ## CPC - G06V 20/42: {of sport video content} - G06N 3/08: Learning methods - G06V 20/63: {Scene text, e.g. street names} - G06V 20/70: Labelling scene content, e.g. deriving syntactic or semantic representations - G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16) - H04N 21/2353: {specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata} - H04N 21/251: {Learning process for intelligent management, e.g. learning user preferences for recommending movies (details of learning user preferences for the retrieval of video data in a video database G06F16/739; computer systems using learning methods G06N3/08)} - H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 21/81: Monomedia components thereof - H04N 21/8133: {specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video programme} - G06V 20/44: {Event detection} - G06V 30/10: Character recognition - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A computing system identifies broadcast video data for a game. The computing system generates tracking data for the game from the broadcast video data using computer vision techniques. The tracking data includes coordinates of players during the game. The computing system generates optical character recognition data for the game from the broadcast video data by applying one or more optical character recognition techniques to each frame of the plurality of frames to extract score and time information from a scoreboard displayed in each frame. The computing system detects a plurality of events that occurred in the game by applying one or more machine learning techniques to the tracking data. The computing system receives play-by-play data for the game. The computing system generates enriched tracking data. The generating includes merging the play-by-play data with one or more of the tracking data, the optical character recognition data, and the plurality of events. ## Claims ### Claim 1 A method, comprising: identifying, by a computing system, broadcast video data for a game, wherein the broadcast video data comprises a plurality of frames; generating, by the computing system, tracking data for the game from the broadcast video data using one or more computer vision techniques, wherein the tracking data comprises coordinates of players during the game; generating, by the computing system, optical character recognition data for the game from the broadcast video data by applying one or more optical character recognition techniques to each frame of the plurality of frames to extract score and time information from a scoreboard displayed in each frame; detecting, by the computing system, a plurality of events that occurred in the game by applying one or more machine learning techniques to the tracking data; automatically detecting, via a neural network, one or more additional events in each frame of a respective broadcast video; enhancing, by the computing system, the detected one or more additional events with contextual information, the contextual information comprising matchup information including one or more matchups; generating, by the computing system, an influence score for each matchup, wherein the influence score captures an influence of a player on a respective matchup; receiving, by the computing system, play-by-play data for the game, wherein the play-by-play data is of a different source type than the broadcast video data; and generating, by the computing system, enriched tracking data, the generating comprising merging the play-by-play data with one or more of the influence score, the tracking data, the optical character recognition data, and the plurality of events. ### Claim 2 (depends on 1) The method of claim 1, further comprising: merging the play-by-play data with the broadcast video data prior to generating the tracking data. ### Claim 3 (depends on 1) The method of claim 1, wherein generating, by the computing system, the enriched tracking data comprises: correcting erroneous outputs in the tracking data or the plurality of events based on information in the play-by-play data. ### Claim 4 (depends on 1) The method of claim 1, wherein generating, by the computing system, the enriched tracking data comprises: segmenting the tracking data into a plurality of possessions; and matching portions of the play-by-play data to the plurality of possessions based on at least one of the tracking data and the plurality of events. ### Claim 5 (depends on 1) The method of claim 1, wherein generating, by the computing system, the enriched tracking data comprises: refining player and ball precisions in each frame of a respective broadcast video. ### Claim 6 A non-transitory computer readable medium comprising one or more sequences of instructions, which, when executed by a processor, causes a computing system to perform operations comprising: identifying, by the computing system, broadcast video data for a game, wherein the broadcast video data comprises a plurality of frames; generating, by the computing system, tracking data for the game from the broadcast video data using one or more computer vision techniques, wherein the tracking data comprises coordinates of players during the game; generating, by the computing system, optical character recognition data for the game from the broadcast video data by applying one or more optical character recognition techniques to each frame of the plurality of frames to extract score and time information from a scoreboard displayed in each frame; detecting, by the computing system, a plurality of events that occurred in the game by applying one or more machine learning techniques to the tracking data; automatically detecting, via a neural network, one or more additional events in each frame of a respective broadcast video; enhancing, by the computing system, the detected one or more additional events with contextual information, the contextual information comprising matchup information including one or more matchups; generating, by the computing system, an influence score for each matchup, wherein the influence score captures an influence of a player on a respective matchup; receiving, by the computing system, play-by-play data for the game, wherein the play-by-play data is of a different source type than the broadcast video data; and generating, by the computing system, enriched tracking data, the generating comprising merging the play-by-play data with one or more of the influence score, the tracking data, the optical character recognition data, and the plurality of events. ### Claim 7 (depends on 6) The non-transitory computer readable medium of claim 6, wherein generating, by the computing system, the enriched tracking data comprises: correcting erroneous outputs in the tracking data or the plurality of events based on information in the play-by-play data. ### Claim 8 (depends on 6) The non-transitory computer readable medium of claim 6, wherein generating, by the computing system, the enriched tracking data comprises: segmenting the tracking data into a plurality of possessions; and matching portions of the play-by-play data to the plurality of possessions based on at least one of the tracking data and the plurality of events. ### Claim 9 (depends on 6) The non-transitory computer readable medium of claim 6, wherein generating, by the computing system, the enriched tracking data comprises: refining player and ball precisions in each frame of a respective broadcast video. ### Claim 10 A system comprising: a processor; and a memory having programming instructions stored thereon, which, when executed by the processor, causes the system to perform operations comprising: identifying broadcast video data for a game, wherein the broadcast video data comprises a plurality of frames; generating tracking data for the game from the broadcast video data using one or more computer vision techniques, wherein the tracking data comprises coordinates of players during the game; generating optical character recognition data for the game from the broadcast video data by applying one or more optical character recognition techniques to each frame of the plurality of frames to extract score and time information from a scoreboard displayed in each frame; detecting a plurality of events that occurred in the game by applying one or more machine learning techniques to the tracking data; automatically detecting, via a neural network, one or more additional events in each frame of a respective broadcast video; enhancing the detected one or more additional events with contextual information, the contextual information comprising matchup information including one or more matchups; generating an influence score for each matchup, wherein the influence score captures an influence of a player on a respective matchup; receiving play-by-play data for the game, wherein the play-by-play data is of a different source type than the broadcast video data; and generating enriched tracking data, the generating comprising merging the play-by-play data with one or more of the influence score, the tracking data, the optical character recognition data, and the plurality of events. ### Claim 11 (depends on 10) The system of claim 10, wherein generating the enriched tracking data comprises: correcting erroneous outputs in the tracking data or the plurality of events based on information in the play-by-play data. ### Claim 12 (depends on 10) The system of claim 10, wherein generating the enriched tracking data comprises: segmenting the tracking data into a plurality of possessions; and matching portions of the play-by-play data to the plurality of possessions based on at least one of the tracking data and the plurality of events. ### Claim 13 (depends on 10) The system of claim 10, wherein generating the enriched tracking data comprises: refining player and ball precisions in each frame of a respective broadcast video.