Systems and methods for computer vision and machine-learning based form feedback

A method implemented by a computing system including one or more processors and storage media storing machine-readable instructions, wherein the method is performed using the one or more processors, the method comprising:

• capturing IR video of a user and exercise equipment being used by the user through a plurality of movements of an exercise;
• generating, at least partially based on the captured IR video, a 3D model of the user including a representation of the user and the exercise equipment being used by the user through the plurality of movements of the exercise;
• estimating a set of joints of the user in the 3D model;
• tracking, based on the estimated set of joints of the user in the 3D model, the user's motion over a period of time;
• determining, based on the user's tracked motion over the period of time and one or more exercise models, a number of repetitions of the exercise performed by the user over the period of time;
• determining, based on the user's tracked motion over the period of time and one or more form feedback models, a form feedback value from a set of form feedback values;
• calculating, based on the number of repetitions and the form feedback value, a user exercise score;
• providing, via a graphical user interface, the user exercise score and the form feedback value to the user, thereby instructing the user to adjust their form during subsequent repetitions of the exercise.