# US 11,113,535 B2 · Determining tactical relevance and similarity of video sequences > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,113,535 B2 - Document ID: US11113535B2 - Status: Granted - Kind: B2 - Application number: 16/678,520 - Filed: 2019-11-08 - Published: 2021-05-13 - Granted: 2021-09-07 - Earliest claimed: 2019-11-08 - Assignees: Second Spectrum, Inc. - Applicants: Second Spectrum, Inc. - Inventors: Vivek Jayaram; Nikhil Mitra; Kevin Squire - Web page: https://patentgrep.com/patent/US11113535B2 ## CPC - H04N 21/23418: {involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - G02B 3/005: {arranged along a single direction only, e.g. lenticular sheets (G02B3/0043 takes precedence)} - G06F 18/22: Matching criteria, e.g. proximity measures - G06V 10/761: {Proximity, similarity or dissimilarity measures} - G06V 20/42: {of sport video content} - G06V 20/46: {Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames} - G06V 20/48: {Matching video sequences} - G09G 3/003: {to produce spatial visual effects} - H04N 21/21805: {enabling multiple viewpoints, e.g. using a plurality of cameras} - H04N 21/2187: Live feed - H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - A61B 3/0041: {characterised by display arrangements} - G06V 20/44: {Event detection} - G06V 2201/10: Recognition assisted with metadata - G09G 2380/08: Biomedical applications - H04N 5/72: Modifying the appearance of television pictures by optical filters or diffusing screens - H04N 13/302: for viewing without the aid of special glasses, i.e. using autostereoscopic displays - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00) - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) ## Abstract Methods and systems for finding and ranking video sequences that contain tactical sequences (e.g., an uninterrupted portion of a sporting event) that are similar to tactical event content in a target video sequence, such as those occurring in sports, include learning and labelling (e.g., through use of metadata and the like) tactically significant sequences in a plurality of video segments, including a plurality of such tactically significant sequences in a single video segment. ## Claims ### Claim 1 A method comprising: receiving a first video feed, capturing a first filmed occurrence, the first video feed comprising a sequence of video frames captured by a camera, wherein the first video feed is a video feed that is consumable by a client device; detecting at least one game chance sequence from the sequence of video frames based on spatiotemporal analysis that identifies game chance boundary events; determining at least one semantic label and associated location data of at least one tactically relevant event in the at least one game chance sequence; concatenating the at least one semantic label and the associated location data as a first labeled tactical sequence; and identifying from a library of stored game chance sequences at least one stored game chance sequence that is similar to the at least one game chance sequence from the first video feed based on a tactical similarity of a stored labeled tactical sequence of the at least one stored game chance sequence with the first labeled tactical sequence. ### Claim 2 (depends on 1) The method of claim 1, wherein the tactical similarity is further based on similarity of a semantic label for the stored labeled tactical sequence and the associated location data with the at least one semantic label and the associated location data of the first labeled tactical sequence. ### Claim 3 (depends on 1) The method of claim 1, wherein a spatiotemporal pattern analysis of the at least one game chance sequence from the sequence of video frames and of the at least one stored game chance sequence indicates a substantive difference in spatiotemporal patterns therein. ### Claim 4 (depends on 1) The method of claim 1, wherein identifying the game chance boundary events includes detecting actions that occur between labeled tactical sequences. ### Claim 5 (depends on 4) The method of claim 4, wherein a game chance boundary event includes at least one of a game dock stoppage, a scoring attempt, a scoring deflection, an event occurring in a region of a field of play, or an event occurring outside of a field of play. ### Claim 6 (depends on 1) The method of claim 1, wherein the game change boundary events facilitate partitioning the first video feed into one or more labeled tactical subsequences. ### Claim 7 (depends on 1) The method of claim 1, wherein the tactical similarity of two sequences comprises each of the two sequences occurring proximal to a common location. ### Claim 8 (depends on 1) The method of claim 1, wherein a location characteristic of a first sequence that is determined by a first level of tactical relevance and similarity computing informs a second level of tactical relevance and similarity computing performing tactical similarity comparison of the first sequence with at least one other sequence. ### Claim 9 (depends on 1) The method of claim 1, wherein the tactical similarity of two sequences is responsive to outcomes of the two sequences. ### Claim 10 (depends on 9) The method of claim 9, wherein the two sequences that produce different outcomes are not tactically similar. ### Claim 11 (depends on 1) The method of claim 1, wherein the tactical similarity is determined by use of a Vantage Point (VP) tree algorithm that applies a distance metric based at least in part on one or more tactical characteristics of the first labeled tactical sequence and of the stored labeled tactical sequence. ### Claim 12 (depends on 1) The method of claim 1, wherein the tactical similarity is determined by use of multi-dimensional distance metric that is based on a plurality of tactical characteristics of the first labeled tactical sequence and of the stored labeled tactical sequence. ### Claim 13 (depends on 1) The method of claim 1, wherein the tactical similarity is determined by application of distance metric values for a plurality of tactical sequence characteristics via a Vantage Point (VP) tree algorithm. ### Claim 14 (depends on 13) The method of claim 13, wherein the plurality of tactical sequence characteristics includes a player identity characteristic. ### Claim 15 A method comprising: receiving a first video feed capturing a first filmed occurrence, the first video feed comprising a sequence of video frames captured by a camera, wherein the first video feed is a video feed that is consumable by a client device; processing the captured first video feed with a first tactical relevance and similarity computing circuit, wherein the first tactical relevance and similarity computing circuit determines, during processing, portions of the video feed that comprise location-event data for one or more semantic events; accessing with a second tactical relevance and similarity computing circuit a tactical similarity support data structure; and determining similarity of the one or more semantic events with tactical video sequences in a library of video sequences by determining for a plurality of tactical similarity characteristic types accessed in the tactical similarity support data structure a concurrence of instances of each of the plurality of tactical similarity characteristic types associated with at least one semantic event of the one or more semantic events and corresponding instances of tactical similarity characteristics associated with the tactical video sequences in the library of video sequences. ### Claim 16 (depends on 15) The method of claim 15, wherein the location-event data indicates at least one of a play, a part of a chance, an event, or a chance demarcation.