# US 11,310,418 B2 · Computer-implemented method for automated detection of a moving area of interest in a video stream of field sports with a common object of interest > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,310,418 B2 - Document ID: US11310418B2 - Status: Granted - Kind: B2 - Application number: 16/962,582 - Filed: 2019-01-18 - Published: 2020-12-24 - Granted: 2022-04-19 - Earliest claimed: 2018-01-19 - Assignees: Veo Technologies Aps - Applicants: Veo Technologies Aps - Inventors: Jesper Molbech Taxbol; Henrik Bjorn Teisbæk; Keld Reinicke - Web page: https://patentgrep.com/patent/US11310418B2 ## CPC - H04N 5/222: Studio circuitry; Studio devices; Studio equipment (cameras or camera modules comprising electronic image sensors, or control thereof H04N23/00) - G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) - G06T 7/33: using feature-based methods - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI] - G06V 20/42: {of sport video content} - G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) - H04N 23/698: for achieving an enlarged field of view, e.g. panoramic image capture - H04N 23/80: Camera processing pipelines; Components thereof - H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums - H04N 23/951: by using two or more images to influence resolution, frame rate or aspect ratio - H04N 23/957: Light-field or plenoptic cameras or camera modules - G06T 2207/10016: Video; Image sequence - G06T 2207/20081: Training; Learning - G06T 2207/30221: Sports video; Sports image - G06T 2207/30224: Ball; Puck - G06T 2207/30241: Trajectory - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A computer-implemented method is provided for automated detection of a moving area of interest, such as a ball, in a video stream of filed sports with a common object of interest encompassing a plurality of players and the object of interest. Images of a sports ground are captured by means of a video camera system producing a video stream which is digitally processed to continuously identify a detected concentration of action within the boundaries of the field. The concentration of action in the field is determined based on an estimated position of the object of interest in at least one frame of the video stream. Players' postures or orientations may be detected to improve the accuracy of the determination of the object of interest. ## Claims ### Claim 1 A computer-implemented method for automated detection of a moving area of interest in a video stream of field sports with a common object of interest encompassing a plurality of players and the object of interest, the method comprising the steps of: (i) image capturing a sports ground by means of a video camera system to produce the video stream; (ii) digitally processing the video stream to continuously identify the area of interest of the game within the boundaries of the sports ground, wherein the area of interest in the sports ground is determined on the basis of an estimated position of the object of interest in at least one frame of the video stream; wherein the position of the object of interest is determined by use of a computer vision and machine learning module; wherein the players' orientations and/or postures in the video stream are detected on the basis of a probability map of the position of the object of interest relative to each player. ### Claim 2 (depends on 1) The method according to claim 1, wherein the machine learning module is configured to detect the players' orientations and/or postures in the video stream. ### Claim 3 (depends on 1) The method according to claim 1, wherein the machine learning module is configured to carry out at least one of the following steps: detect the ball in the video stream; detect the players in the video stream; detect one or more predetermined events based on the players' position in the video stream; detect the area of interest in the video stream. ### Claim 4 (depends on 3) The method according to claim 3, wherein the machine learning module outputs a single channel image presenting the probability of the object of interest being present in an input frame to the machine learning module. ### Claim 5 (depends on 4) The method according to claim 4, further comprising the step of updating the probability of the position of the object of interest in the frame, and representing the particles in a digital output frame or in a stream of digital output frames, and automatically estimating the position of the object of interest on the basis of a detected density of a cloud of particles. ### Claim 6 (depends on 3) The method according to claim 3, wherein the detection of players is carried out on the basis of: pre-stored data representing the position of the video camera system relative to the sports ground; and pre-stored information data representing characteristics of the players. ### Claim 7 (depends on 6) The method according to claim 6, wherein the players are tracked in the digital video stream by use of linear quadratic estimation. ### Claim 8 (depends on 1) The method according to claim 1, wherein estimated position of the object of interest is determined on the basis of a probability representation of the object of interest in the at least one frame of the video stream. ### Claim 9 (depends on 1) The method according to claim 1, wherein at least 100 particles are modelled in the at least one frame of the video stream to determine each particle's probability of representing the object of interest. ### Claim 10 (depends on 1) The method according to claim 1, wherein: the machine learning module accesses and is thus trained using a database of uniformly sized pictures of players, each picture including one player, wherein in each of the pictures the position of the object of interest relative to the player is known; and a probability of a direction of the object of interest is produced on runtime for each player. ### Claim 11 (depends on 10) The method according to claim 10, further comprising the step of updating the probability of the position of the object of interest in the frame, and representing the particles in a digital output frame or in a stream of digital output frames, and automatically estimating the position of the object of interest on the basis of a detected density of a cloud of particles; wherein the probabilities of the presence of the object of interest as estimated are combined with said possible position of the object of interest relative to the player to estimate the position of the object of interest relative to the player. ### Claim 12 (depends on 1) The method according to claim 1, wherein the concentration of action in the object of interest is detected on the basis of object of interest positions in a plurality of respective frames of the panoramic video stream captured between a first point in time and a second point in time. ### Claim 13 (depends on 1) The method according to claim 1, wherein the concentration of action in the sports ground is detected on the basis of the detection of the players' positions relative to the sports ground in the plurality of frames of the panoramic video stream captured between a first point in time and a second point in time. ### Claim 14 A computer-implemented method for automated detection of a moving area of interest in a video stream of field sports with a common object of interest encompassing a plurality of players and the object of interest, the method comprising the steps of: (i) image capturing a sports ground by means of a video camera system to produce the video stream; (ii) digitally processing the video stream to continuously identify the area of interest of the game within the boundaries of the sports ground, wherein the area of interest in the sports ground is determined on the basis of an estimated position of the object of interest in at least one frame of the video stream; wherein the position of the object of interest is determined by use of a computer vision and machine learning module; wherein at least 100 particles are modelled in the at least one frame of the video stream to determine each particle's probability of representing the object of interest; wherein the step of modelling the particles comprises: projecting each particle onto a point in each one of the at least one frame; sampling the output of the machine learning module at that point in the said frame, onto which the particle is projected; and updating the position of each particle by use of Newtonian dynamics.