System and method for automated and feedback based human movement guidance

A system for providing automated feedback on user movements compared to pre-recorded instructor movements, the system comprising:

• a motion capture system configured to record movements of an instructor and generate three-dimensional (3D) motion capture data;
• a computing device comprising a processor and a memory, the memory storing instructions that, when executed by the processor, cause the computing device to:
• receive the 3D motion capture data of the instructor movements;
• assign the 3D motion capture data to a hyper-realistic avatar of the instructor;
• preprocess the 3D motion capture data to generate a plurality of artificial intelligence (AI) predictions from different angles and room conditions;
• a user device comprising a camera, a display, and an application, wherein the application is configured to:
• capture a camera feed of a user performing movements;
• identify, in real-time, a skeletal frame of the user from the camera feed;
• compare the user's skeletal frame to the instructor's preprocessed AI predictions using a neural network to determine if the user's movements match the instructor's movements;
• display, on the user device, the user's skeletal frame compared to the instructor's movements;
• generate, based on the comparison, real-time feedback to the user via voice commands, indicating when the user's movements are out of sync or position compared to the instructor's movements;
• and
• provide, at an end of a session, analytics on the user's performance compared to the instructor's movements,
• wherein the motion capture system comprises at least one of an optical motion capture system, an inertial motion capture system, or a marker less motion capture system, wherein:
• the optical motion capture system utilizes reflective markers worn by the instructor and a plurality of cameras configured to track positions of the reflective markers;
• the inertial motion capture system utilizes a plurality of inertial measurement units worn by the instructor and configured to track the instructor's movements;
• and the marker less motion capture system utilizes a plurality of depth sensing cameras configured to track the instructor's movements without the use of worn sensors or markers.