Method and system for automatic extraction of virtual on-body inertial measurement units

A method of operating an automated processing pipeline comprising:

• determining, via a two-dimensional skeletal estimator, skeletal-associated points of a body of a person in a plurality of frames of a video data set;
• generating, via a three-dimensional skeletal estimator, 3D motion estimation of 3D joints of the skeletal-associated points;
• determining, via an IMU extractor, motion values at one or more 3D joints of the skeletal-associated points;
• modifying, via a sensor emulator, the determine motion values at one or more 3D joints of the skeletal-associated points according to an IMU sensor profile to generate virtual IMU sensor values;
• and
• outputting the virtual IMU sensor values for the one or more 3D joints of the skeletal-associated points.

A non-transitory computer-readable medium having instructions stored thereon, wherein execution of the instructions by a processor, cause the processor to:

• determine skeletal-associated points of a body of a person in a plurality of frames of a video data set;
• generate 3D motion estimation of 3D joints of the skeletal-associated points;
• determine motion values at one or more 3D joints of the skeletal-associated points;
• modify the determined motion values at one or more 3D joints of the skeletal-associated points according to an IMU sensor profile to generate virtual IMU sensor values;
• and
• output the virtual IMU sensor values for the one or more 3D joints of the skeletal-associated points.