# US 2023/0056531 A1 · Methods and Systems for Utilizing Live Embedded Tracking Data within a Live Sports Video Stream > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2023/0056531 A1 - Document ID: US20230056531A1 - Status: Published application - Kind: A1 - Application number: 17/821,002 - Filed: 2022-08-19 - Published: 2023-02-23 - Earliest claimed: 2021-08-20 - Assignees: Stats LLC - Applicants: Stats LLC - Inventors: Sateesh Pedagadi; Johannes Kuehnel; Bradford Griffiths; Christian Marko; Raphael Reiners; Brian Orefice; Patrick Joseph Lucey - Web page: https://patentgrep.com/patent/US20230056531A1 ## CPC - H04N 21/23614: {Multiplexing of additional data and video streams (arrangements for simultaneous broadcast of plural pieces of information H04H20/28)} - A63F 13/355: Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client - A63F 13/60: Generating or modifying game content before or while executing the game program, e.g. authoring tools specially adapted for game development or game-integrated level editor - H04N 21/2187: Live feed - H04N 21/23418: {involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 21/43074: {of additional data with content streams on the same device, e.g. of EPG data or interactive icon with a TV programme} - H04N 21/4312: {involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations} - H04N 21/4348: {Demultiplexing of additional data and video streams} - H04N 21/4884: {for displaying subtitles} - H04N 21/8133: {specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video programme} - A63F 13/812: Ball games, e.g. soccer or baseball - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR ## Abstract A computing system receives a video stream of a game. The computing system generates tracking data corresponding to the video stream using one or more artificial intelligence models. The computing system generates interactive video data by combining the video stream of the game with the tracking data. The computing system causes a media player to render graphics corresponding to the tracking data over the video stream by sending the interactive video data to a client device executing the media player. ## Claims ### Claim 1 A method comprising: receiving, by a computing system, a video stream of a game; generating, by the computing system, tracking data corresponding to the video stream using one or more artificial intelligence models; generating, by the computing system, interactive video data by combining the video stream of the game with the tracking data; and causing, by the computing system, a media player to render graphics corresponding to the tracking data over the video stream by sending the interactive video data to a client device executing the media player. ### Claim 2 (depends on 1) The method of claim 1, wherein generating, by the computing system, the interactive video data by combining the video stream of the game with the tracking data comprises: multiplexing the video stream with the tracking data to generate a multiplexed data stream. ### Claim 3 (depends on 2) The method of claim 2, wherein multiplexing the video stream with the tracking data to generate the multiplexed data stream comprises: mapping the tracking data with the video stream on a packet-by-packet basis; and adding the mapping of the tracking data with the video stream to the video stream as a subtitle track within the video stream. ### Claim 4 (depends on 1) The method of claim 1, wherein generating, by the computing system, the tracking data corresponding to the video stream using the one or more artificial intelligence models comprises: generating a JavaScript Object Notation (JSON) file comprising the tracking data. ### Claim 5 (depends on 4) The method of claim 4, causing, by the computing system, the media player to render the graphics corresponding to the tracking data over the video stream comprises: causing the client device to unpack the JSON file to identify metadata events in the tracking data. ### Claim 6 (depends on 1) The method of claim 1, wherein generating, by the computing system, the tracking data corresponding to the video stream using the one or more artificial intelligence models comprises: generating player and ball positional data corresponding to locational coordinates of each player and the ball. ### Claim 7 (depends on 1) The method of claim 1, wherein generating, by the computing system, the tracking data corresponding to the video stream using the one or more artificial intelligence models comprises: generating event data based on the tracking data, wherein the event data captures events that occur within the game. ### Claim 8 A non-transitory computer readable medium comprising one or more sequences of instructions, that, when executed by one or more processors, causes a computing system to perform operations comprising: receiving, by the computing system, a video stream of a game; generating, by the computing system, tracking data corresponding to the video stream using one or more artificial intelligence models; generating, by the computing system, interactive video data by combining the video stream of the game with the tracking data; and causing, by the computing system, a media player to render graphics corresponding to the tracking data over the video stream by sending the interactive video data to a client device executing the media player. ### Claim 9 (depends on 8) The non-transitory computer readable medium of claim 8, wherein generating, by the computing system, the interactive video data by combining the video stream of the game with the tracking data comprises: multiplexing the video stream with the tracking data to generate a multiplexed data stream. ### Claim 10 (depends on 9) The non-transitory computer readable medium of claim 9, wherein multiplexing the video stream with the tracking data to generate the multiplexed data stream comprises: mapping the tracking data with the video stream on a packet-by-packet basis; and adding the mapping of the tracking data with the video stream to the video stream as a subtitle track within the video stream. ### Claim 11 (depends on 8) The non-transitory computer readable medium of claim 8, wherein generating, by the computing system, the tracking data corresponding to the video stream using the one or more artificial intelligence models comprises: generating a JavaScript Object Notation (JSON) file comprising the tracking data. ### Claim 12 (depends on 11) The non-transitory computer readable medium of claim 11, causing, by the computing system, the media player to render the graphics corresponding to the tracking data over the video stream comprises: causing the client device to unpack the JSON file to identify metadata events in the tracking data. ### Claim 13 (depends on 8) The non-transitory computer readable medium of claim 8, wherein generating, by the computing system, the tracking data corresponding to the video stream using the one or more artificial intelligence models comprises: generating player and ball positional data corresponding to locational coordinates of each player and the ball. ### Claim 14 (depends on 8) The non-transitory computer readable medium of claim 8, wherein generating, by the computing system, the tracking data corresponding to the video stream using the one or more artificial intelligence models comprises: generating event data based on the tracking data, wherein the event data captures events that occur within the game. ### Claim 15 A system comprising: a processor; and a memory having programming instructions stored thereon, which, when executed by the processor, causes a computing system to perform operations comprising: receiving a video stream of a game; generating tracking data corresponding to the video stream using one or more artificial intelligence models; generating interactive video data by combining the video stream of the game with the tracking data; and causing a media player to render graphics corresponding to the tracking data over the video stream by sending the interactive video data to a client device executing the media player. ### Claim 16 (depends on 15) The system of claim 15, wherein generating the interactive video data by combining the video stream of the game with the tracking data comprises: multiplexing the video stream with the tracking data to generate a multiplexed data stream. ### Claim 17 (depends on 16) The system of claim 16, wherein multiplexing the video stream with the tracking data to generate the multiplexed data stream comprises: mapping the tracking data with the video stream on a packet-by-packet basis; and adding the mapping of the tracking data with the video stream to the video stream as a subtitle track within the video stream. ### Claim 18 (depends on 15) The system of claim 15, wherein generating the tracking data corresponding to the video stream using the one or more artificial intelligence models comprises: generating a JavaScript Object Notation (JSON) file comprising the tracking data. ### Claim 19 (depends on 18) The system of claim 18, causing the media player to render the graphics corresponding to the tracking data over the video stream comprises: causing the client device to unpack the JSON file to identify metadata events in the tracking data. ### Claim 20 (depends on 15) The system of claim 15, wherein generating the tracking data corresponding to the video stream using the one or more artificial intelligence models comprises: generating player and ball positional data corresponding to locational coordinates of each player and the ball; and generating event data based on the tracking data, wherein the event data captures events that occur within the game.