Video-based tracking systems and methods

A system for video-based tracking of objects or parts of objects in one or more regions of interest, comprising:

• a camera capturing a 2D image stream of a region of interest, the 2D image stream depicting zero or more objects for tracking or detection;
• an image processor to:
• receive the captured 2D image stream from the camera;
• generate a detection data set from the captured 2D image stream, a terrain model of a reference environment model, and a camera calibration, wherein the camera calibration involves determining a position and orientation of the camera with respect to the terrain model, the detection data set having one or more detection data entries of respective 2D imaging stream frame positions for detected objects or object parts in each frame of the captured 2D image stream in the reference environment model, wherein the detection data entries comprise at least one 3D shape and position of the detected objects or object parts relative to the terrain model of the reference environment model determined using an intersection of a line of sight corresponding to a vertex of the detected objects or object parts with the terrain model of the reference environment model;
• a fusion processor configured to:
• generate fused 3D referenced data by fitting the detection data entries to a shape and a trajectory of the detected objects in the terrain model of the reference environment model, wherein the trajectory is modelled as a continuous function over time and over a surface of the terrain model;
• and
• wherein the detected data entries include a detected shape in the terrain model of the reference environment model, and wherein the fitting considers a comparison of the detected shape and a test shape.

A system for video-based tracking of objects or parts of objects in one or more regions of interest, comprising:

• a camera capturing a 2D image stream of a region of interest, the 2D image stream depicting zero or more objects for tracking or detection;
• an image processor to:
• receive the captured 2D image stream from the camera;
• generate a detection data set for the captured 2D image stream, a terrain model of a reference environment model, and a camera calibration, wherein the camera calibration involves determining a position and orientation of the camera with respect to the terrain model, the detection data set having one or more detection data entries of respective 2D imaging stream frame positions for detected objects or object parts in each frame of the captured 2D image stream in the reference environment model, wherein the detection data entries comprise at least one 3D shape and position of the detected objects or object parts relative to the terrain model of the reference environment model determined using an intersection of a line of sight corresponding to a vertex of the detected objects or object parts with the terrain model of the reference environment model;
• a fusion processor configured to:
• generate fused 3D referenced data by fitting the detection data entries to a shape and a trajectory of the detected objects in the terrain model of the reference environment model, wherein the trajectory is modelled as a continuous function over time and over a surface of the terrain model;
• wherein the fusion processor determines the most likely shape and trajectory among the set of test shapes and test trajectories by assigning a likelihood to test trajectories and test shapes, wherein the likelihood is inversely related to acceleration.