# US 10,441,866 B2 · Method and system for determining whether a spherical element impacts with a component of a playing field, or arranged on or proximate thereto > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,441,866 B2 - Document ID: US10441866B2 - Status: Granted - Kind: B2 - Application number: 15/918,595 - Filed: 2018-03-12 - Published: 2018-07-19 - Granted: 2019-10-15 - Earliest claimed: 2013-04-17 - Assignees: Foxtenn Bgreen, S.L. - Applicants: Foxtenn Bgreen, S. L. - Inventors: Javier Simón Vilar - Web page: https://patentgrep.com/patent/US10441866B2 ## CPC - A63B 71/0605: {Decision makers and devices using detection means facilitating arbitration} - A63B 69/38: for tennis {(A63B61/006, A63B69/0073 and A63B69/0097 take precedence)} - G06V 20/42: {of sport video content} - G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) - A63B 71/0622: {Visual, audio or audio-visual systems for entertaining, instructing or motivating the user} - A63B 2024/0034: {during flight} - A63B 2024/0043: {Systems for locating the point of impact on a specific surface} - A63B 2071/0611: {Automatic tennis linesmen, i.e. in-out detectors} - A63B 2071/0638: {Displaying moving images of recorded environment, e.g. virtual environment} - A63B 2220/14: Geo-tagging, e.g. for correlating route or track location data with specific information related to that specific location - A63B 2220/805: Optical or opto-electronic sensors - A63B 2220/806: Video cameras - A63B 2220/808: Microphones - A63B: APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT (apparatus for passive exercising, massage A61H) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A method and system for determining whether a spherical element impacts with a component of a playing field, or arranged on or proximate thereto. The method includes acquiring images of a surveillance area of a field that covers at least part of said component, such as a delimiting perimeter line of a game area or a target, performing an approximate detection of an impact of a spherical element relative to that component or proximate thereto, with an object detection and recognition system that can discern when the detected object is indeed a spherical element, automatically selecting one of the images acquired for the same point in time and that includes the area where said impact has occurred, and analyzing the selected image to check if the spherical element has impacted or not with the component. ## Claims ### Claim 1 A method for determining whether a spherical element impacts with a playing field, or with a component arranged on said playing field or proximate to said playing field, said component comprising one of a physical structure, physical marking, or virtual marking, and said playing field being made up of game areas, where the method comprises the steps of: a) acquiring, with detection devices comprising at least a plurality of high speed cameras, a sequence of images of at least one area of surveillance of said playing field covering at least part of at least said component and storing the sequence of images on a memory; b) performing, by a processor of a computer system operatively connected with said detection devices, when the detected object is indeed a spherical element, an approximate detection of the impact of the spherical element with said component or proximate thereto; c) automatically selecting, by the processor in response to said approximate detection, at least one image, from said sequence of images, acquired for a point in time that includes the at least one area of surveillance of the playing field where a bounce takes place and when the bounce takes place; and d) determining, with the processor, that the spherical element has impacted with said component; wherein: prior to said approximate detection of the impact of step b), a pre-detection of the passage of the spherical element through at least two points located at different heights is performed with at least the detection devices, and based on said pre-detection, a prediction of the trajectory to be followed by the spherical element towards the ground and a predetermination or predefinition of an approximate area where the impact or bounce will occur is performed by the processor, said pre-detection being performed using at least two detection devices. ### Claim 2 (depends on 1) The method according to claim 1, wherein said detection and said pre-detection of the impact are done by the same plurality of high speed cameras. ### Claim 3 (depends on 1) The method according to claim 1, wherein said detection means further include sound detection devices which are operatively connected to said processor. ### Claim 4 (depends on 3) The method according to claim 3 wherein said prediction of the passage of the spherical element through at least two points located at different heights is further performed with the assistance of a three-dimensional type object detection and recognition system including at least scanners located at different heights. ### Claim 5 (depends on 3) The method according to claim 3, wherein the scanners are laser scanners, and the three dimensional type object detection and recognition system further comprises ultrasound devices, and the method comprises performing said automatic discerning using said laser scanners to obtain distance information, with respect to said laser scanners, from each point of contact of the laser beams emitted by the laser scanners with the detected object, and based on said distance information obtained for a group of contact points. ### Claim 6 (depends on 1) The method according to claim 1 wherein in said step a) said acquisition of images only covers several targets within the area of surveillance of said playing field. ### Claim 7 (depends on 1) The method according to claim 1, wherein said approximate detection of step b) comprises performing at least two detections that include at least a moment when said impact or bounce happens and/or at least a moment immediately before the impact or bounce and/or at least a moment immediately after the impact or bounce, using in said step c), as the time point for the automatic selection of said at least one image, at least one of the time points of said at least two detections. ### Claim 8 (depends on 1) The method according to claim 1, wherein said step d) comprises showing said at least one selected image on displaying means to allow performing the aforementioned analysis by visually judging the image, or to supplement said analysis. ### Claim 9 (depends on 1) The method according to claim 1, wherein: said step c) comprises selecting a plurality of images from said sequence of images and arranged in sequence, wherein the plurality of selected images includes the at least one image acquired for the point in time of the approximate detection and images for points in time before and after that point; and said step d) comprises a video playback including said plurality of images, in slow motion forward and/or backwards and/or stopping in order to perform the aforementioned visual judgment. ### Claim 10 (depends on 1) The method according to claim 1, comprising performing said step a) the plurality of high speed cameras covering the various components of the playing field, or arranged on or proximate thereto, from different sides thereof, and/or the different sections of the perimeter lines from both sides and/or ends thereof and/or all the targets from different sides thereof. ### Claim 11 (depends on 1) The method according to claim 1, comprising using said predefinition of said approximate area where the bounce will occur to activate and/or control one or more high-speed cameras whose coverage area includes the approximate predefined area. ### Claim 12 (depends on 1) The method according to claim 1, comprising automatically discerning when the detected object is indeed a spherical element based on the difference in the detection signals caused by being detected by the detection and recognition system in comparison with the detection signals resulting from the detection of other elements, stationary or mobile, of at least different shape and size. ### Claim 13 (depends on 1) The method according to claim 1, wherein said spherical element is a tennis ball and said playing field is a tennis court, with said game areas being the different rectangular areas of said tennis court. ### Claim 14 (depends on 1) The method according to claim 1, wherein said spherical element is a tennis ball and said playing field is a tennis court, with said game areas being the different rectangular areas of said tennis court and wherein the method comprises selecting as perimeter lines on which check when the tennis ball has really bounced or not, the following: those demarcating a respective service area to which a player will serve or has served, to judge whether the tennis ball bounces in or out of the service area; or those demarcating the total game area for a single or a doubles match during the rally after the service. ### Claim 15 (depends on 14) The method according to claim 14, comprising receiving information on whether, after the service, the tennis ball has touched a net or not of the playing field, and based on this information, after judging whether the tennis ball bounces in or out of the service area, the method comprises issuing a signal indicating that: the serve must be repeated, when the ball has bounced inside the service area but after touching the net, or the serve must not be repeated, when the ball has bounced outside the service area after touching the net.