# US 9,999,825 B2 · Smart-court system and method for providing real-time debriefing and training services of sport games

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 9,999,825 B2
- Document ID: US9999825B2
- Status: Granted
- Kind: B2
- Application number: 14/370,006
- Filed: 2013-02-21
- Published: 2015-01-15
- Granted: 2018-06-19
- Earliest claimed: 2012-02-23
- Assignees: Playsight Interactive Ltd.
- Applicants: Playsight Interactive Ltd.
- Inventors: Chen Shachar; Evgeni Khazanov; Yoram Ben Zur
- Web page: https://patentgrep.com/patent/US9999825B2

## CPC

- A63B 71/02: for large-room or outdoor sporting games
- A63B 71/04: for small-room or indoor sporting games
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 7/292: Multi-camera tracking
- G06V 20/42: {of sport video content}
- A63B 2225/50: Wireless data transmission, e.g. by radio transmitters or telemetry
- A63B: APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT (apparatus for passive exercising, massage A61H)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A Smart-court system, adaptive to constrained sport environment, for enabling real time analysis and debriefing of sport activities is provided herein. The Smart-court system is comprised of: (i) an automatic recording system comprising a plurality of video cameras located in a court, arranged to real-time (RT) recording of a sport session and utilizing automatic calibration and stabilization module; and (ii) a data processing system comprising: (a) a capture module for grabbing a video stream; (b) an objects' detector module arranged to extract during the RT sport session, the objects from the foreground of each frame; (c) an event module for automatically analyzing, the motion and the activities of the tracked objects for automatically identifying and classifying events, creating a synchronized event log and calculating statistics that occurred during the RT sport session; and (d) a presentation module enabling to perform instant debriefing, combined biomechanical and tactical analysis of the video.

## Claims

### Claim 1

A Smart-court system, comprising:
an automatic recording system comprising a plurality of video cameras, arranged for capturing a sport session in a video stream, and
a data processing system comprising a computer programmed to implement:
a capture module for receiving the video stream of the sport session;
an object detector module, for extracting simultaneously to said receiving of the video stream of the sport session, a plurality of objects from frames of the video stream, thereby tracking the objects appearing in the video stream;
an event module for automatically analyzing, during the sport session, motion and activities of the tracked objects, the motion and activities comprising at least two interactions between a ball and an object, for automatically identifying events and classifying the identified events into types based on said tracking and analysis, and for creating a synchronized event log of the classified events based on said identifying and classifying; and
a presentation module for enabling a user to combine biomechanical and tactical analysis of the video stream by selectively playing at least one of the identified events classified into the types, by type, based on the synchronized event log.

### Claim 2 (depends on 1)

The smart court system of claim 1, further comprising a tracking module configured to identify in real time a part of the sport session, the part starting with a stroke and ending when a player gains a score.

### Claim 3 (depends on 1)

The Smart-court system of claim 1, wherein the computer is further configured to upload and publish an online video associated with the synchronized event log on a social network.

### Claim 4 (depends on 1)

The Smart-court system of claim 1, wherein one of the tracked objects is a player and the event module is further configured by programming the computer, to yield a pattern of behavior and a tactical behavior of the player from the identified events.

### Claim 5 (depends on 4)

The Smart-court system of claim 4, wherein the computer is further programmed to generate a profile of the player, the profile comprising information based on the yielded tactical behavior and the yielded pattern of behavior.

### Claim 6 (depends on 1)

The Smart-court system of claim 1, wherein said automatic recording system is further configured to perform automatic calibration by dividing a court as appearing in the video stream using at least one grid, and in each frame of the video stream, checking the grid for deviation of the court lines from the grid.

### Claim 7 (depends on 1)

The Smart-court system of claim 1, wherein the computer is further configured to identify a start and end of a sport game or training and to automatically identify a change of status of the sport game.

### Claim 8 (depends on 1)

The Smart-court system of claim 1, wherein the video that is transmitted from the cameras is automatically customized to adjust to environment conditions and does not require an intervention of an operator.

### Claim 9 (depends on 1)

The Smart-court system of claim 1, wherein vibration in the stream of video is automatically reduced.

### Claim 10 (depends on 1)

The Smart-court system of claim 1, wherein the data processing module further enables to create automatic customized video movie by integrating event or segments according to given criteria which are determined by the user or predefined by the system.

### Claim 11 (depends on 1)

The Smart-court system of claim 1, wherein the computer is further programmed to identify a change of location of players when the players switch sides on a court used by the players during the sport session.

### Claim 12 (depends on 1)

The Smart-court system of claim 1, further comprising an additional techniques and strategies module configured to provide recommendations to a player on how to correct activity in a sport game based on said analyzing.

### Claim 13 (depends on 1)

The Smart-court system of claim 1, wherein said computer is further configured to use said identifying of the events, for cutting out idle time in the video stream.

### Claim 14 (depends on 1)

The Smart-court system of claim 1, wherein said presentation module is further configured by programming, to enable playing an abridged video displaying only some of the identified events by jumping between some of the identified events according to a user query and the synchronized log event.

### Claim 15 (depends on 1)

The Smart-court system of claim 1, wherein the presentation module is further configured by programming to play a virtual 3D dynamic scene using a 3D engine which simulates objects behavior based on said analyzing and the synchronized event log.

### Claim 16 (depends on 1)

The Smart-court system of claim 1, wherein said computer is further configured to identify occurrence of a mental event based on said analyzing.

### Claim 17

A method for implementing a smart court system, the method comprising the steps of
capturing a sport session in a video stream using a plurality of video cameras;
receiving the video stream of the sport session on a computer;
simultaneously to said receiving, extracting a plurality of objects from frames of the video stream, thereby tracking the objects appearing in the video stream;
during the sport session, automatically analyzing motion and activities of the tracked objects that appear in the stream of video, the motion and activities comprising at least two interactions between a ball and an object;
automatically identifying events and classifying the events into types based on the tracking and analysis of the motion and the activities; and
creating a synchronized event log of the classified events based on said identifying and classifying, wherein a user is enabled to combine biomechanical and tactical analysis of the video stream by selectively playing at least one of the identified events classified into types, by type, using the synchronized event log.

### Claim 18 (depends on 17)

The method of claim 17, further comprising identifying in real time, a part of the sport session, the part starting with a stroke and ending when a player gains a score.

### Claim 19 (depends on 17)

The method of claim 17, wherein the method further comprises playing an abridged video by jumping between some of the identified events based on a user query and according to the synchronized event log.

### Claim 20 (depends on 17)

The method of claim 17, further comprising publishing and uploading game statistics, the online video and an associated event log in one or more social networks.

### Claim 21 (depends on 17)

The method of claim 17, further comprising yielding a pattern of behavior and a tactical behavior of a player from more than one of the identified events.

### Claim 22 (depends on 21)

The method of claim 21, further comprising generating and uploading to a social network, a profile of the player, the profile comprising information based on the yielded tactical behavior and the yielded pattern of behavior.

### Claim 23 (depends on 17)

The method of claim 17, further comprising identifying a start and end of a sport game or training and automatically identifying a change of status of the sport game.

### Claim 24 (depends on 17)

The method of claim 17, further comprising automatically customizing the video that is transmitted from the cameras to adjust to environment conditions and does not require an intervention of an operator.

### Claim 25 (depends on 17)

The method of claim 17, further comprising automatically reducing vibration in the stream of video.

### Claim 26 (depends on 17)

The method of claim 17, further comprising identifying the change of status by a predefined gesture of the user.

### Claim 27 (depends on 17)

The method of claim 17, further comprising identifying change of designated location of players, when the players switch sides on the court.

### Claim 28 (depends on 17)

The method of claim 17, further comprising providing recommendations to a player on how to correct activity in a sport game based on said analyzing.

### Claim 29 (depends on 17)

The method of claim 17, further comprising cutting out idle time in the video file to save storage space using said identifying of the events.

### Claim 30 (depends on 17)

The method of claim 17, further comprising playing an abridged video displaying only some of the identified events by jumping between some of the identified events according to a user query and the synchronized log event.

### Claim 31 (depends on 17)

The method of claim 17, further comprising playing a virtual 3D dynamic scene based on the video stream, by using a 3D engine to simulate objects behavior based on said analyzing and the synchronized event log.

### Claim 32 (depends on 17)

The method of claim 17, further comprising identifying occurrence of a mental event based on said analyzing.

### Claim 33 (depends on 17)

The method of claim 17, further comprising creating customized video movie by integrating events or segments according to given criteria which are determined by the user or predefined by the system.

### Claim 34 (depends on 17)

The method of claim 17, further comprising identifying unforced error based on identifying timing and position of a player before striking the ball.

### Claim 35 (depends on 17)

The method of claim 17, further comprising real time automatic calculation of game score based on said identified and classified events and an analysis of player and ball position and motion.

### Claim 36 (depends on 17)

The method of claim 17, further comprising comparing between the players' profiles and providing performance matching between players.