# US 2026/0087649 A1 · Method and System for Optically Tracking Moving Objects

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0087649 A1
- Document ID: US20260087649A1
- Status: Published application
- Kind: A1
- Application number: 19/327,680
- Filed: 2025-09-12
- Published: 2026-03-26
- Earliest claimed: 2024-09-20
- Assignees: Topgolf Sweden AB
- Applicants: Topgolf Sweden AB
- Inventors: Dennis Johansson
- Web page: https://patentgrep.com/patent/US20260087649A1

## CPC

- G01S 13/726: {Multiple target tracking}
- G06T 7/248: {involving reference images or patches}
- G01S 13/723: {by using numerical data}
- G01S 13/867: {Combination of radar systems with cameras}
- G01S 13/89: for mapping or imaging
- G06T 7/251: {involving models}
- G06T 7/277: involving stochastic approaches, e.g. using Kalman filters
- G06T 7/292: Multi-camera tracking
- G06V 20/42: {of sport video content}
- A63B 2024/0034: {during flight}
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/30224: Ball; Puck
- G06T 2207/30241: Trajectory
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- A63B: APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT (apparatus for passive exercising, massage A61H)

## Abstract

Method for tracking an object moving through a three-dimensional space within a field of view of a digital camera, includes: obtaining from a radar directional, radial distance and/or speed information regarding a hypothetical moving objects; determining, based on the information and without taking into consideration information from the digital camera, two hypothetical three-dimensional radar-based trajectories that are transformed into image plane coordinates hypothetical two-dimensional radar-based trajectories; obtaining a series of images of the three-dimensional space, that are analyzed to detect a two-dimensional camera-based trajectory; mapping, in the image plane coordinates, the two-dimensional camera-based trajectory to one of the hypothetical two-dimensional radar-based trajectories; and determining a three-dimensional trajectory based on the two-dimensional camera-based trajectory in combination with hypothetical two-dimensional radar-based trajectory, but without considering directional information associated with the mapped hypothetical two-dimensional radar-based trajectory.

## Claims

### Claim 1

Method for tracking an object moving through a three-dimensional space within a field of view of a digital camera, comprising:
obtaining from a radar directional information regarding one or more hypothetical moving objects moving through the three-dimensional space in relation to the radar, as well as radial distance and/or speed information regarding the one or more hypothetical moving objects in relation to the radar;
determining, based on the obtained information and without taking into consideration information from the digital camera, two or more hypothetical three-dimensional radar-based trajectories;
transforming each of the hypothetical three-dimensional radar-based trajectories into image plane coordinates specific to the digital camera to achieve a set of two or more hypothetical two-dimensional radar-based trajectories;
obtaining from the digital camera a series of consecutively captured images representing optical input from the three-dimensional space;
analyzing the series of consecutively captured images to detect a two-dimensional camera-based trajectory of the object through the field of view;
mapping, in the image plane coordinates, the two-dimensional camera-based trajectory to one of the hypothetical two-dimensional radar-based trajectories; and
determining a three-dimensional trajectory of the object through the three-dimensional space based on the two-dimensional camera-based trajectory in combination with radial distance and/or speed information associated with the mapped hypothetical two-dimensional radar-based trajectory, but without considering directional information associated with the mapped hypothetical two-dimensional radar-based trajectory.

### Claim 2 (depends on 1)

Method according to claim 1, comprising:
updating the two-dimensional camera-based trajectory based on additional captured images from the digital camera; and
iteratively tracking the object by determining the three-dimensional trajectory of the object based on the updated two-dimensional camera-based trajectory and regarding the radial distance and/or speed information associated with the hypothetical two-dimensional radar-based trajectory, without considering updated information regarding the directional information associated with the hypothetical two-dimensional radar-based trajectory.

### Claim 3 (depends on 2)

Method according to claim 2, comprising:
identifying an inability to detect updated information regarding the two-dimensional camera-based trajectory;
as a result of the identifying of the inability and from a time of the identifying moving forward, iteratively tracking the object based on updated information from the radar, the iterative tracking comprising:
receiving from the radar updated radial distance and/or speed information, and also updated directional information; and
updating the three-dimensional trajectory of the object based on the updated information.

### Claim 4 (depends on 3)

Method according to claim 3, comprising:
identifying a renewed ability to detect updated information regarding the two-dimensional camera-based trajectory; and
as a result of the identifying of the renewed ability and from a time of the identifying moving forward, again iteratively tracking the object by determining the three-dimensional trajectory of the object based on updated information regarding the two-dimensional camera-based trajectory and regarding the radial distance and/or speed information associated with the hypothetical two-dimensional radar-based trajectory, without considering updated information regarding the directional information associated with the hypothetical two-dimensional radar-based trajectory.

### Claim 5

Method for tracking an object moving through a three-dimensional space within a field of view of a digital camera, comprising:
obtaining from a radar directional information regarding one or more hypothetical moving objects moving through the three-dimensional space in relation to the radar, as well as radial distance and/or speed information regarding the one or more hypothetical moving objects in relation to the radar;
determining, based on the obtained information and without taking into consideration information from the digital camera, two or more hypothetical three-dimensional radar-based trajectories;
transforming each of the hypothetical three-dimensional radar-based trajectories into image plane coordinates specific to the digital camera to determine a set of regions of interest each corresponding to a location in the consecutively captured images of a respective one of the one or more hypothetical moving objects;
obtaining from the digital camera a series of consecutively captured images representing optical input from the three-dimensional space;
analyzing the series of consecutively captured images to detect, only in the set of regions of interest, at least one blob;
mapping, in the image plane coordinates, the blob to one of the hypothetical three-dimensional radar-based trajectories; and
determining a three-dimensional trajectory of the object through the three-dimensional space based on the image location of the blob in combination with the radial distance and/or speed information associated with the one hypothetical three-dimensional radar-based trajectory, but without considering directional information associated with the one hypothetical three-dimensional radar-based trajectory.

### Claim 6 (depends on 5)

Method according to claim 5, comprising
without considering information from the digital camera, and before the operation of transforming, adjusting or filtering out at least one of the two or more hypothetical three-dimensional radar-based trajectories based on a physics model of the corresponding hypothetical object travelling through the three-dimensional space.

### Claim 7 (depends on 5)

Method according to claim 5, comprising:
identifying an inability to map a two-dimensional camera-based trajectory or blob detected in the consecutively captured series of consecutively captured images to any one of the hypothetical radar-based trajectories; and
as a result of the identifying of the inability, determining the three-dimensional trajectory of the object without taking into consideration updated information from the radar.

### Claim 8 (depends on 5)

Method according to claim 5, comprising:
storing in a memory not all of the series of consecutively captured images, but a most recent set of the series of consecutively captured images;
transforming at least one of the hypothetical three-dimensional radar-based trajectories into the image plane coordinates to determine a region of interest corresponding to a location in the stored consecutively captured images; and
identifying a two-dimensional camera-based trajectory of the object in connection to the region of interest and across the stored series of consecutively captured images.

### Claim 9 (depends on 5)

Method according to claim 5, wherein
the storing of the series of consecutively captured images in memory comprises storing the series of consecutively captured images in a circular buffer.

### Claim 10 (depends on 5)

Method according to claim 5, comprising:
determining, for two or more of the consecutively captured images, a set of one or more blobs;
correlating the set of one or more blobs to each other across the two or more of the consecutively captured images, to form a set of one or more hypothetical camera-based trajectories for one or more hypothetical objects; and
determining the three-dimensional trajectory of the object based on one of the one or more hypothetical camera-based trajectories.

### Claim 11 (depends on 10)

Method according to claim 10, comprising:
applying a physics model to determine credibility of the hypothetical camera-based trajectories; and
adjusting or filtering out one or more of the hypothetical camera-based trajectories based on the determined credibility.

### Claim 12 (depends on 11)

Method according to claim 11, wherein
the physics model is a three-dimensional physics model, and wherein
the hypothetical camera-based trajectories are hypothetical three-dimensional camera-based trajectories.

### Claim 13 (depends on 5)

Method according to claim 5, comprising:
obtaining from the radar, at two or more different points in time, directional information regarding the one or more hypothetical objects moving through the three-dimensional space in relation to the radar, as well as radial distance and/or speed information regarding the one or more hypothetical moving objects in relation to the radar, to form hypothetical radar-based object information; and
correlating the hypothetical radar-based object information across the two or more different points in time to form the two or more hypothetical three-dimensional radar-based trajectories.

### Claim 14 (depends on 13)

Method according to claim 13, comprising:
applying a physics model to determine credibility of the one or more hypothetical three-dimensional radar-based trajectories; and
adjusting or filtering out one or more of the hypothetical three-dimensional radar-based trajectories based on the determined credibility.

### Claim 15

System for tracking an object moving through a three-dimensional space within a field of view of a digital camera, the system comprising:
a radar information analyzer configured to obtain, from a radar, directional information regarding one or more hypothetical moving objects moving through the three-dimensional space in relation to the radar, as well as radial distance and/or speed information regarding the one or more hypothetical moving objects in relation to the radar; and
a digital image analyzer configured to obtain, from the digital camera, a series of consecutively captured images representing optical input from the three-dimensional space,
the system being configured to
determine, based on the obtained information and without taking into consideration information from the digital camera, two or more hypothetical three-dimensional radar-based trajectories;
transform each of the hypothetical three-dimensional radar-based trajectories into image plane coordinates specific to the digital camera to achieve a set of two or more hypothetical two-dimensional radar-based trajectories;
analyze the series of consecutively captured images to detect a two-dimensional camera-based trajectory of the object through the field of view;
map, in the image plane coordinates, the two-dimensional camera-based trajectory to one of the hypothetical two-dimensional radar-based trajectories; and
determine a three-dimensional trajectory of the object through the three-dimensional space based on the two-dimensional camera-based trajectory in combination with radial distance and/or speed information associated with the mapped hypothetical two-dimensional radar-based trajectory, but without considering directional information associated with the mapped hypothetical two-dimensional radar-based trajectory.

### Claim 16

Non-transitory computer-readable medium encoding instructions for tracking an object moving through a three-dimensional space within a field of view of a digital camera, the instructions configured to, when executing on one or more processors, perform operations comprising:
obtaining from a radar directional information regarding one or more hypothetical moving objects moving through the three-dimensional space in relation to the radar, as well as radial distance and/or speed information regarding the one or more hypothetical moving objects in relation to the radar;
determining, based on the obtained information and without taking into consideration information from the digital camera, two or more hypothetical three-dimensional radar-based trajectories;
transforming each of the hypothetical three-dimensional radar-based trajectories into image plane coordinates specific to the digital camera to achieve a set of two or more hypothetical two-dimensional radar-based trajectories;
obtaining from the digital camera a series of consecutively captured images representing optical input from the three-dimensional space;
analyzing the series of consecutively captured images to detect a two-dimensional camera-based trajectory of the object through the field of view;
mapping, in the image plane coordinates, the two-dimensional camera-based trajectory to one of the hypothetical two-dimensional radar-based trajectories; and
determining a three-dimensional trajectory of the object through the three-dimensional space based on the two-dimensional camera-based trajectory in combination with radial distance and/or speed information associated with the mapped hypothetical two-dimensional radar-based trajectory, but without considering directional information associated with the mapped hypothetical two-dimensional radar-based trajectory.

### Claim 17 (depends on 1)

Method according to claim 1, comprising:
without considering information from the digital camera, and before the operation of transforming, adjusting or filtering out at least one of the two or more hypothetical three-dimensional radar-based trajectories based on a physics model of the corresponding hypothetical object travelling through the three-dimensional space.

### Claim 18 (depends on 1)

Method according to claim 1, comprising:
identifying an inability to map a two-dimensional camera-based trajectory or blob detected in the consecutively captured series of consecutively captured images to any one of the hypothetical radar-based trajectories; and
as a result of the identifying of the inability, determining the three-dimensional trajectory of the object without taking into consideration updated information from the radar.

### Claim 19 (depends on 1)

Method according to claim 1, comprising:
storing in a memory not all of the series of consecutively captured images, but a most recent set of the series of consecutively captured images;
transforming at least one of the hypothetical three-dimensional radar-based trajectories into the image plane coordinates to determine a region of interest corresponding to a location in the stored consecutively captured images; and
identifying a two-dimensional camera-based trajectory of the object in connection to the region of interest and across the stored series of consecutively captured images.

### Claim 20 (depends on 1)

Method according to claim 1, comprising:
determining, for two or more of the consecutively captured images, a set of one or more blobs;
correlating the set of one or more blobs to each other across the two or more of the consecutively captured images, to form a set of one or more hypothetical camera-based trajectories for one or more hypothetical objects; and
determining the three-dimensional trajectory of the object based on one of the one or more hypothetical camera-based trajectories.
