# US 2022/0277463 A1 · Tracking dynamics using a computerized device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0277463 A1
- Document ID: US20220277463A1
- Status: Published application
- Kind: A1
- Application number: 17/638,073
- Filed: 2020-08-26
- Published: 2022-09-01
- Earliest claimed: 2019-08-26
- Assignees: Agt International GmbH
- Applicants: Agt International GmbH
- Inventors: Markus SCHLATTMANN; Thomas BADER; Robert BIEHL; Philipp HUELSDUNK; Paul IDSTEIN; Michael LEIGSNERING; Sergey SUKHANOV
- Web page: https://patentgrep.com/patent/US20220277463A1

## CPC

- A63F 13/655: by importing photos, e.g. of the player
- G06T 7/251: {involving models}
- A63F 13/213: comprising photodetecting means, e.g. cameras, photodiodes or infrared cells (A63F13/219 takes precedence)
- A63F 13/812: Ball games, e.g. soccer or baseball
- G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments
- G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics
- H04N 23/61: based on recognised objects
- H04N 23/951: by using two or more images to influence resolution, frame rate or aspect ratio
- A63F 13/573: using trajectories of game objects, e.g. of a golf ball according to the point of impact
- A63F 13/577: using determination of contact between game characters or objects, e.g. to avoid collision between virtual racing cars
- G06T 2207/10024: Color image
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30196: Human being; Person
- G06T 2207/30224: Ball; Puck
- G06T 2207/30241: Trajectory
- G06T 2210/21: Collision detection, intersection
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

The presently disclosed subject matter there includes a computerized device that comprises a processing circuitry, and one or more sensors including at least one image sensor configured to continuously capture images of an environment; the processing circuitry is configured to process the captured images and: detect an object within the environment; track a trajectory of movement of the object; select a best fitting motion model to the trajectory and determine a state of the object based on the best fitting motion model; and determine at least one intersection point between the object and a target object.

## Claims

### Claim at position 1

-34. (canceled)

### Claim 35

A computerized device comprising a processing circuitry, the processing circuitry comprising at least one computer processor; the computerized device further comprises or is otherwise operatively connected to one or more sensors including at least one camera configured to continuously capture images of an environment;
the processing circuitry is configured to process the captured images and:
detect a moving object within the environment;
track a trajectory of movement of the moving object comprising:
determine a plurality of location-points, each location-point is determined based on one or more detected appearances of the object in the captured images;
determine moving object parameters;
determine the trajectory of the moving object, based on the plurality of location-points and parameters; select a best fitting motion model to the trajectory from a plurality of available motion models, and determine a state of the object based on the best fitting motion model; and
determine at least one intersection point between the object and a target object.

### Claim 36 (depends on 35)

The computerized device of claim 35, wherein the processing circuitry is further configured to:
continuously update the trajectory of the moving object, using additional location-points, to thereby generate an updated trajectory;
compare the updated trajectory with a plurality of motion models to determine a best fitting motion model to the updated trajectory;
identify a change in the state of the object, in case it is determined that a motion model that is different from the selected motion model has a best fit to the updated trajectory.

### Claim 37 (depends on 35)

The computerized device of claim 35, wherein the processing circuitry is configured for determining the trajectory of the moving object based on the plurality of location-points and parameters, to:
define a sliding window encompassing a sub-group of the plurality of location-points;
shift the position of the sliding window along the plurality of location points;
for each window position:
determine, based on a respective sub-group of location-points in the window, a selected motion model having a best fit to a sub-section of the trajectory consisting of the sub-group of location-points;
assign a current state to the moving object according to the selected motion model determined for the plurality of windows.

### Claim 38 (depends on 37)

The computerized device of claim 37, wherein the processing circuitry is configured to reapply the selected motion model on the trajectory.

### Claim 39 (depends on 35)

The computerized device of claim 35, wherein the processing circuitry is configured for determining the at least one intersection point between the object and the target, to:
estimate a plurality of future trajectories of the moving object at time of intersection with the target object, thereby obtaining a plurality of candidate intersection points;
calculate a distribution of the plurality of candidate intersection points; and
determine a final intersection point if a distance of the object from the target is less than a predefmed distance and a variance of the distribution is less than a predefmed value.

### Claim 40 (depends on 35)

The computerized device of claim 35, wherein the processing circuitry is further configured to:
generate an activity world as a virtual environment that corresponds to a real world physical environment, the activity world defining an area where an object's dynamics are being monitored;
define an activity area having specific boundaries within the activity world, the activity area being assigned with one or more attributes defming constraints on a manner in which an activity of a user should be performed from within the activity area;
detect according to the at least one intersection point an interaction of the user with the object, the interaction resulting in movement of the object in the environment;
following detection of interaction of the user with the object:
determine an area of interaction within the activity world;
determine whether the area of interaction overlaps with a predefined activity area, and if so:
determine whether interaction attributes comply with one or more respective attributes assigned to the predefined activity area; and
determine a level of success in performing a user activity.

### Claim 41 (depends on 40)

The computerized device of claim 40, wherein the processing circuitry is configured, following determination of at least one intersection point between the object and the target object, to:
determine whether the at least one intersection point, complies with the one or more attributes and determine a level of success in performing the interaction accordingly.

### Claim 42 (depends on 35)

The computerized device of claim 35, wherein processing of the images is performed in real-time while the images are being captured.

### Claim 43 (depends on 35)

The computerized device of claim 35, wherein the processing circuitry is configured to adapt a camera frame rate according to the state of the object.

### Claim 44

A method of monitoring using activity in an environment using a computerized device, comprising a processing circuitry that includes or is otherwise operatively connected to at least one image sensing device; the method comprising:
capturing images of the environment using the at least one image sensing device;
processing the captured images, comprising:
detecting an object within the environment;
tracking a trajectory of movement of the object comprising;
determining a plurality of location-points, each location-point is determined based on one or more detected appearances of the object in the captured images;
determining moving object parameters;
determining a trajectory of the moving object based on the plurality of location-points and parameters;
selecting a best fitting motion model to the trajectory from a plurality of available motion models, and determining a state of the object based on the best fitting motion model; and
determining at least one intersection point between the object and a target object.

### Claim 45 (depends on 44)

The method of claim 44, further comprising:
continuously updating the trajectory of the object using additional location-points, to thereby generate an updated trajectory;
comparing the updated trajectory with a plurality of motion models to determine a best fitting motion model to the updated trajectory;
identifying a change in the state of the object, in case it is determined that a motion model that is different from the selected motion model has a best fit to the updated trajectory.

### Claim 46 (depends on 44)

The method of claim 44 further comprising, for determining the trajectory of the moving object based on the plurality of location-points and parameters:
defining a sliding window encompassing a sub-group of the plurality of location-points;
shifting the position of the sliding window along the plurality of location points;
for each window position:
determining, based on a respective sub-group of location-points in the window, a selected motion model having a best fit to a sub-section of the trajectory consisting of the sub-group of location-points; and
assigning a current state to the moving object according to the selected motion model determined for the plurality of windows.

### Claim 47 (depends on 44)

The method of claim 44 further comprising, for determining the at least one intersection point between the object and the target:
estimating a plurality of future trajectories of the moving object at a time of intersection with the target object, thereby obtaining a plurality of candidate intersection points;
calculating a distribution of the plurality of candidate intersection points;
and determining a fmal intersection point if a distance of the object from the target is less than a predefmed distance and a variance of the distribution is less than a predefmed value.

### Claim 48 (depends on 44)

The method of claim 44 further comprising:
generating an activity world as a virtual environment that corresponds to a real world physical environment, the activity world defming an area where an object's dynamics are being monitored;
defining an activity area having specific boundaries within the activity world, the activity area being assigned with one or more attributes defining constraints on a manner in which an activity of a user should be performed from within the activity area;
detecting an interaction of the user with the object, the interaction resulting in movement of the object in the environment; and
following detection of interaction of the user with the object:
determining an area of interaction within the activity world;
determining whether the area of interaction overlaps with a predefmed activity area, and if so:
determining whether interaction attributes comply with one or more respective attributes assigned to the predefmed activity area; and
determining level of success in performing a user activity.

### Claim 49 (depends on 48)

The method of claim 48 further comprising following determination of at least one intersection point between the object and the target object:
determining whether the at least one intersection point, complies with the one or more attributes, and determining a level of success in performing the interaction accordingly.

### Claim 50 (depends on 44)

The method of claim 44 wherein processing of the images is performed in real-time while the images are being captured.

### Claim 51 (depends on 44)

The method of claim 44 further comprising adapting a camera frame rate according to the state of the object.

### Claim 52

A non-transitory computer readable storage medium tangibly embodying a program of instructions that, when executed by a computer, cause the computer to perform a method. of monitoring using activity in an environment using a computerized device, comprising a processing circuitry that includes or is otherwise operatively connected to at least one image sensing device; the method comprising:
capturing images of the environment using the at least one image sensing device;
processing the captured images, comprising:
detecting an object within the environment;
tracking a trajectory of movement of the object comprising;
determining a plurality of location-points, each location-point is determined based on one or more detected appearances of the object in the captured images;
determining moving object parameters;
determining a trajectory of the moving object based on the plurality of location-points and parameters;
selecting a best fitting motion model to the trajectory from a plurality of available motion models, and determining a state of the object based on the best fitting motion model; and
determining at least one intersection point between the object and a target object.
