Computer system for automatically analyzing a video of a physical activity using a model and providing corresponding feedback

A method comprising:

• receiving, by one or more processors, a video file including plurality of image frames showing an entity performing a physical activity that involves a sequence of movement phases;
• generating, by the one or more processors, a plurality of coordinate sets by performing image analysis on the plurality of image frames, each coordinate set of the plurality of coordinate sets corresponding to an image frame of the plurality of image frames and including a plurality of coordinates of a plurality of physical features of an entity depicted in the image frame;
• providing, by the one or more processors, the plurality of coordinate sets as input to a trained model, the trained model being configured to assign a plurality of scores and a plurality of movement phases to the plurality of image frames based on the plurality of coordinate sets by, for each coordinate set of the plurality of coordinate sets:
• executing a forward algorithm based on the coordinate set associated with a current image frame to determine a plurality of joint probability values, the plurality of joint probability values including one or more joint probability values associated with one or more previous image frames, and the plurality of joint probability values including a joint probability value associated with the current image frame;
• determining a likelihood metric associated with the current image frame based on the plurality of joint probability values;
• determining a score for the current image frame based on a previous likelihood metric associated with a previous image frame of the plurality of image frames and the likelihood metric associated with the current image frame, the score being one of the plurality of scores;
• executing a decoding algorithm based on the coordinate set to identify a movement phase that is most likely depicted in the current image frame, the movement phase being one of the movement phases in the sequence;
• and
• assigning the score and the movement phase to the current image frame;
• selecting, by the one or more processors, a particular movement phase in the sequence of movement phases for which to provide feedback based on the plurality of scores and the plurality of movement phases assigned to the plurality of image frames;
• and
• generating, by the one or more processors, the feedback for the entity about their performance of the particular movement phase based on a set of predefined rules, the feedback including guidance for improving future performance of the particular movement phase as compared to the performance shown in the video file.

A non-transitory computer-readable medium comprising program code that is executable by one or more processors for causing the one or more processors to:

• receive a video file including plurality of image frames showing an entity performing a physical activity that involves a sequence of movement phases;
• generate a plurality of coordinate sets by performing image analysis on the plurality of image frames, each coordinate set of the plurality of coordinate sets corresponding to an image frame of the plurality of image frames and including a plurality of coordinates of a plurality of physical features of an entity depicted in the image frame;
• provide the plurality of coordinate sets as input to a trained model, the trained model being configured to assign a plurality of scores and a plurality of movement phases to the plurality of image frames based on the plurality of coordinate sets by, for each coordinate set of the plurality of coordinate sets:
• executing a forward algorithm based on the coordinate set associated with a current image frame to determine a plurality of joint probability values, the plurality of joint probability values including one or more joint probability values associated with one or more previous image frames, and the plurality of joint probability values including a joint probability value associated with the current image frame;
• determining a likelihood metric associated with the current image frame based on the plurality of joint probability values;
• determining a score for the current image frame based on a previous likelihood metric associated with a previous image frame of the plurality of image frames and the likelihood metric associated with the current image frame, the score being one of the plurality of scores;
• executing a decoding algorithm based on the coordinate set to identify a movement phase that is most likely depicted in the current image frame, the movement phase being one of the movement phases in the sequence;
• and
• assigning the score and the movement phase to the current image frame;
• select a particular movement phase in the sequence of movement phases for which to provide feedback based on the plurality of scores and the plurality of movement phases assigned to the plurality of image frames;
• and
• generate the feedback for the entity about their performance of the particular movement phase based on a set of predefined rules, the feedback including guidance for improving future performance of the particular movement phase as compared to the performance shown in the video file.