# US 12,182,714 B2 · Methods for detecting events in sports using a convolutional neural network > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,182,714 B2 - Document ID: US12182714B2 - Status: Granted - Kind: B2 - Application number: 16/254,128 - Filed: 2019-01-22 - Published: 2019-07-25 - Granted: 2024-12-31 - Earliest claimed: 2018-01-21 - Assignees: Stats LLC - Applicants: Stats LLC - Inventors: Paul Power; Jennifer Hobbs; Patrick Lucey - Web page: https://patentgrep.com/patent/US12182714B2 ## CPC - G06N 3/084: Backpropagation, e.g. using gradient descent - G06F 18/23: Clustering techniques - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/048: Activation functions - G06N 3/08: Learning methods - G06N 3/09: Supervised learning - G06N 20/00: Machine learning - G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments - G06V 10/762: using clustering, e.g. of similar faces in social networks - G06V 10/764: using classification, e.g. of video objects - G06V 10/82: using neural networks - G06V 20/42: {of sport video content} - G11B 27/28: by using information signals recorded by the same method as the main recording {(G11B27/22 takes precedence)} - G06T 2207/30196: Human being; Person - G06T 2207/30228: Playing field - H04L 67/10: in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract A method of identifying a defensive alignment and an offensive alignment in a set-piece is disclosed herein. A computing system receives one or more streams of tracking data. The computing system identifies a set-piece contained in the one or more streams of tracking data. The computing system identifies a defensive alignment of a first team and an offensive alignment of a second team. The computing system extracts, via a convolutional neural network, one or more features corresponding to a type of defensive alignment implemented by the first team by passing the set-piece through the convolutional neural network. The computing system scans the set-piece, via a machine learning algorithm, to identify one or more features indicative of a type of offensive alignment implemented by the second team. The computing system infers the type of defensive alignment implemented by the first team. ## Claims ### Claim 1 A method of identifying a team alignment during a set-piece, comprising: receiving, by a computing system, one or more streams of tracking data associated with one or more matches; identifying, by the computing system, one or more set-piece plays contained in the one or more streams of the tracking data, wherein the one or more set-piece plays include a portion of the one or more matches where play is started following a stoppage or re-started following the stoppage; identifying, by the computing system, a defensive alignment of a first team by: extracting, by the computing system via a trained convolutional neural network, one or more features corresponding to a type of defensive alignment implemented by the first team by passing each of the set-piece plays through the trained convolutional neural network, wherein the one or more features include one or more local edge patterns, one or more interactions, or one or more player locations; identifying, by the computing system, an offensive alignment of a second team by: scanning each of the one or more set-piece plays, by the computing system via the trained convolutional neural network, to identify one or more features indicative of a type of offensive alignment implemented by the second team; determining, by the computing system, a success outcome for each of the one or more set-piece plays; generating, by the computing system, a first graphical representation of the defensive alignment or the offensive alignment and a corresponding effectiveness, wherein the effectiveness is based on the success outcome for each of the one or more set-piece plays, and wherein the effectiveness indicates whether the defensive alignment was successful or whether the offensive alignment was successful; and generating, by the computing system, a second graphical representation that includes a cluster of an average team behavior for the first team or the second team, wherein the average team behavior is determined based on the extracting and the scanning, wherein the convolutional neural network is trained by: retrieving a set of match data from a data store, identifying, in each match, one or more set-piece plays, extracting player motion before and after each set-piece play, and learning one or more features unique to each offensive and defensive alignment found in the set of match data. ### Claim 2 (depends on 1) The method of claim 1, wherein receiving, by the computing system, the one or more streams of the tracking data, comprises: receiving one or more streams of live tracking data from a tracking system located in a remote venue. ### Claim 3 (depends on 1) The method of claim 1, wherein identifying, by the computing system, the one or more set-piece plays contained in the one or more streams of the tracking data, comprises: extracting a subset of tracking data from the one or more streams of the tracking data, wherein the subset of tracking data comprises agent motion before and after the one or more set-piece plays. ### Claim 4 (depends on 1) The method of claim 1, wherein the tracking data comprises one or more images of the one or more matches. ### Claim 5 (depends on 1) The method of claim 1, wherein the one or more streams of the tracking data are image-based representations of the one or more matches. ### Claim 6 (depends on 1) The method of claim 1, the identifying, by the computing system, the defensive alignment of the first team further comprising: determining, by the computing system, one or more unique identifiers based on the tracking data; and extracting, by the computing system via the trained convolutional neural network, the one or more features corresponding to the type of defensive alignment based on the one or more unique identifiers. ### Claim 7 A system for identifying a defensive alignment and an offensive alignment in a set-piece, comprising: a processor; and a memory having programming instructions stored thereon, which, when executed by the processor, performs one or more operations comprising: receiving one or more streams of tracking data associated with one or more matches; identifying one or more set-piece plays contained in the one or more streams of the tracking data, wherein the one or more set-piece plays include a portion of the one or more matches where play is started following a stoppage or re-started following the stoppage; identifying a defensive alignment of a first team by: extracting, via a trained convolutional neural network, one or more features corresponding to a type of defensive alignment implemented by the first team by passing each of the set-piece plays through the trained convolutional neural network, wherein the one or more features include one or more local edge patterns, one or more interactions, or one or more player locations; identifying an offensive alignment of a second team by: scanning each of the one or more set-piece plays, via the trained convolutional neural network, to identify one or more features indicative of a type of offensive alignment implemented by the second team; determining a success outcome for each of the one or more set-piece plays; generating a first graphical representation of the defensive alignment or the offensive alignment and a corresponding effectiveness, wherein the effectiveness is based on the success outcome for each of the one or more set-piece plays, and wherein the effectiveness indicates whether the defensive alignment was successful or whether the offensive alignment was successful; and generating a second graphical representation that includes a cluster of an average team behavior for the first team or the second team, wherein the average team behavior is determined based on the extracting and the scanning, wherein the convolutional neural network is trained by: retrieving a set of match data from a data store, identifying, in each match, one or more set-piece plays, extracting player motion before and after each set-piece play, and learning one or more features unique to each offensive and defensive alignment found in the set of match data. ### Claim 8 (depends on 7) The system of claim 7, wherein receiving the one or more streams of the tracking data, comprises: receiving one or more streams of live tracking data from a tracking system located in a remote venue. ### Claim 9 (depends on 7) The system of claim 7, wherein identifying the one or more set-piece plays contained in the one or more streams of the tracking data, comprises: extracting a subset of tracking data from the one or more streams of the tracking data, wherein the subset of tracking data comprises agent motion before and after the one or more set-piece plays. ### Claim 10 (depends on 9) The system of claim 9, further wherein the tracking data comprises one or more images of the one or more matches. ### Claim 11 (depends on 10) The system of claim 10, wherein the one or more streams of the tracking data are image-based representations of the one or more matches. ### Claim 12 (depends on 7) The system of claim 7, the identifying the defensive alignment of the first team further comprising: determining one or more unique identifiers based on the tracking data; and extracting, via the trained convolutional neural network, the one or more features corresponding to the type of defensive alignment based on the one or more unique identifiers. ### Claim 13 A non-transitory computer readable medium including one or more sequences of instructions that, when executed by one or more processors, causes a computing system to perform operations comprising: receiving, by the computing system, one or more streams of tracking data associated with one or more matches; identifying, by the computing system, one or more set-piece plays contained in the one or more streams of the tracking data, wherein the one or more set-piece plays include a portion of the one or more matches where play is started following a stoppage or re-started following the stoppage; identifying, by the computing system, a defensive alignment of a first team by: extracting, by the computing system via a trained convolutional neural network, one or more features corresponding to a type of defensive alignment implemented by the first team by passing each of the set-piece plays through the trained convolutional neural network, wherein the one or more features include one or more local edge patterns, one or more interactions, or one or more player locations; identifying, by the computing system, an offensive alignment of a second team by: scanning each of the one or more set-piece plays, by the computing system via the trained convolutional neural network, to identify one or more features indicative of a type of offensive alignment implemented by the second team; determining, by the computing system, a success outcome for each of the one or more set-piece plays; and generating, by the computing system, a first graphical representation of the defensive alignment or the offensive alignment and a corresponding effectiveness, wherein the effectiveness is based on the success outcome for each of the one or more set-piece plays, and wherein the effectiveness indicates whether the defensive alignment was successful or whether the offensive alignment was successful; and generating, by the computing system, a second graphical representation that includes a cluster of an average team behavior for the first team or the second team, wherein the average team behavior is determined based on the extracting and the scanning, wherein the convolutional neural network is trained by: retrieving a set of match data from a data store, identifying, in each match, one or more set-piece plays, extracting player motion before and after each set-piece play, and learning one or more features unique to each offensive and defensive alignment found in the set of match data. ### Claim 14 (depends on 13) The non-transitory computer readable medium of claim 13, wherein receiving, by the computing system, the one or more streams of tracking data, comprises: receiving one or more streams of live tracking data from a tracking system located in a remote venue. ### Claim 15 (depends on 13) The non-transitory computer readable medium of claim 13, where in identifying, by the computing system, the one or more set-piece plays contained in the one or more streams of the tracking data, comprises: extracting a subset of tracking data from the one or more streams of the tracking data, where in the subset of tracking data comprises agent motion before and after the one or more set-piece plays. ### Claim 16 (depends on 13) The non-transitory computer readable medium of claim 13, wherein the tracking data comprises one or more images of the one or more matches. ### Claim 17 (depends on 13) The non-transitory computer readable medium of claim 13, where in the one or more streams of the tracking data are image-based representations of the one or more matches.