Self-organized learning of three-dimensional motion data

A system for capturing and analyzing motion, comprising a computing system communicatively coupled to one or more cameras and one or more radar sensors, the computing system configured to:

• train a machine learning model in a training environment, to:
• identify and track a moving object based on training image data, and
• identify movement signature of the moving object based on training radar data;
• obtain, by the trained machine learning model, image data associated with the moving object in an analyzed environment at a plurality of points in time from the one or more cameras;
• obtain, by the trained machine learning model, radar data associated with the moving object in the analyzed environment at the plurality of points in time from the one or more radar sensors;
• pair, by the trained machine learning model, each image datum with a corresponding radar datum based on chronological occurrence of the image data and the radar data in the analyzed environment;
• and
• generate, by the trained machine learning model, the three-dimensional motion representation relating to the moving object that is associated with the image data and the radar data;
• wherein a number of physical devices configured to collect training data in the training environment is greater than a number of physical devices in the analyzed environment and wherein the physical devices include at least one camera unit and one radar unit;
• wherein the trained machine learning model is configured to recognize and distinguish between two or more different moving objects based on sensor data obtained from the analyzed environment;
• wherein the trained machine learning model is configured to match the image data of each of the two or more different moving objects with the radar data captured at same points in time with the image data, wherein the trained machine learning model is configured to determine that the radar data corresponding to first one of the two or more different moving objects includes first frequency signature characteristics indicating movement corresponding to the first one of the two or more different moving objects, and that the radar data corresponding to second one of the two or more different moving objects excludes the first frequency signature characteristics.

One or more non-transitory computer-readable storage media configured to store instructions that, in response to being executed, cause a system to perform operations, the operations comprising:

• training a machine learning model in a training environment, to:
• identify and track a moving object based on training image data, and
• identify movement signature of the moving object based on training radar data;
• capturing image data associated with the moving object in an analyzed environment at a plurality of points in time;
• capturing radar data associated with the moving object in the analyzed environment at the plurality of points in time;
• obtaining, by the trained machine learning model, the image data and the radar data associated with the moving object in the analyzed environment;
• pairing each image datum with a corresponding radar datum based on chronological occurrence of the image data and the radar data in the analyzed environment;
• and
• generating, by the trained machine learning model, the three-dimensional motion representation relating to the moving object that is associated with the image data and the radar data;
• wherein a number of physical devices configured to collect training data in the training environment is greater than a number of physical devices in the analyzed environment and wherein the physical devices include at least one camera unit and one radar unit;
• wherein the trained machine learning model is configured to recognize and distinguish between two or more different moving objects based on sensor data obtained from the analyzed environment;
• wherein the trained machine learning model is configured to match the image data of each of the two or more different moving objects with the radar data captured at same points in time with the image data, wherein the trained machine learning model is configured to determine that the radar data corresponding to first one of the two or more different moving objects includes first frequency signature characteristics indicating movement corresponding to the first one of the two or more different moving objects, and that the radar data corresponding to second one of the two or more different moving objects excludes the first frequency signature characteristics.