# US 12,762,896 B2 · System and method for automated and feedback based human movement guidance > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,762,896 B2 - Document ID: US12762896B2 - Status: Granted - Kind: B2 - Application number: 18/659,024 - Filed: 2024-05-09 - Published: 2025-11-13 - Granted: 2026-10-13 - Earliest claimed: 2024-05-09 - Applicants: Mark Hamilton - Inventors: Mark Hamilton - Web page: https://patentgrep.com/patent/US12762896B2 ## CPC - G06V 40/23: {Recognition of whole body movements, e.g. for sport training} - G06T 13/40: of characters, e.g. humans, animals or virtual beings - G06T 15/005: {General purpose rendering architectures} - G06V 40/174: {Facial expression recognition} - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract The present invention relates to a motion capture and analysis method and system that provides automated, personalized feedback to users performing physical movements. The invention allows movement experience leaders to create hyper-realistic digital avatars of themselves, which guide users through specific movements and analyze their performance in real-time using advanced motion capture and AI techniques. By leveraging state-of-the-art game engines, neural networks, and 3D motion capture data, the system enables a highly immersive and interactive learning experience that closely mimics human-to-human instruction. The invention aims to provide a scalable platform for experts to deliver personalized training to a wide audience, with potential applications in virtual fitness classes, dance lessons, and sports coaching. ## Claims ### Claim 1 A system for providing automated feedback on user movements compared to pre-recorded instructor movements, the system comprising: a motion capture system configured to record movements of an instructor and generate three-dimensional (3D) motion capture data; a computing device comprising a processor and a memory, the memory storing instructions that, when executed by the processor, cause the computing device to: receive the 3D motion capture data of the instructor movements; assign the 3D motion capture data to a hyper-realistic avatar of the instructor; preprocess the 3D motion capture data to generate a plurality of artificial intelligence (AI) predictions from different angles and room conditions; a user device comprising a camera, a display, and an application, wherein the application is configured to: capture a camera feed of a user performing movements; identify, in real-time, a skeletal frame of the user from the camera feed; compare the user's skeletal frame to the instructor's preprocessed AI predictions using a neural network to determine if the user's movements match the instructor's movements; display, on the user device, the user's skeletal frame compared to the instructor's movements; generate, based on the comparison, real-time feedback to the user via voice commands, indicating when the user's movements are out of sync or position compared to the instructor's movements; and provide, at an end of a session, analytics on the user's performance compared to the instructor's movements, wherein the motion capture system comprises at least one of an optical motion capture system, an inertial motion capture system, or a marker less motion capture system, wherein: the optical motion capture system utilizes reflective markers worn by the instructor and a plurality of cameras configured to track positions of the reflective markers; the inertial motion capture system utilizes a plurality of inertial measurement units worn by the instructor and configured to track the instructor's movements; and the marker less motion capture system utilizes a plurality of depth sensing cameras configured to track the instructor's movements without the use of worn sensors or markers. ### Claim 2 (depends on 1) The system of claim 1, wherein the instructions, when executed by the processor, further cause the computing device to preprocess the 3D motion capture data to generate the plurality of AI predictions by: rendering, using a game engine, the 3D motion capture data assigned to the instructor avatar from a plurality of virtual camera angles; varying, in the game engine, virtual lighting and environment conditions; capturing, from the game engine, a plurality of 2D frames of the instructor avatar performing the movements; and analyzing the plurality of 2D frames using one or more trained neural networks to generate the AI predictions. ### Claim 3 (depends on 1) The system of claim 1, wherein the neural network used to compare the user's skeletal frame to the instructor's preprocessed AI predictions is trained using a dataset comprising: a plurality of 2D frames of the instructor avatar performing the movements from different angles and under different virtual environment conditions; and corresponding 3D motion capture data of the instructor's movements temporally aligned with the 2D frames. ### Claim 4 (depends on 1) The system of claim 1, wherein the application is further configured to generate the real-time feedback to the user by: displaying, on the user device, a visual indicator for each tracked skeletal key point, wherein the visual indicator is configured to change color based on a calculated deviation between a position of the skeletal key point and a corresponding key point position in the instructor's preprocessed AI predictions. ### Claim 5 (depends on 1) The system of claim 1, wherein the analytics on the user's performance comprise at least one of: a timeline of the user's skeletal frame positions compared to the instructor's movements; a graph of calculated deviations for each skeletal keypoint over the session; a score based on the user's overall accuracy in matching the instructor's movements; or recommendations for improvement based on the user's performance. ### Claim 6 (depends on 1) The system of claim 1, wherein the application is further configured to: transmit the user's performance data to a remote server, wherein the remote server is configured to compare the user's performance data to performance data of a plurality of other users; and display a leaderboard showing the user's performance ranking among the plurality of other users. ### Claim 7 (depends on 1) The system of claim 1, further comprising: an audio capture system configured to record the instructor's voice while recording the instructor's movements, wherein the instructions, when executed by the processor, further cause the computing device to: synchronize the recorded audio with the 3D motion capture data; and play back the recorded audio in synchronization with the instructor avatar's animated movements in the application.