Real-time body pose estimation system and method in unconstrained video

A computer-implemented method for tracking body features of a subject and producing a pose estimation using an unconstrained video sequence at computational speeds approximating real time, the video sequence constituting a physical record of the body features of the subject, the method comprising:

• receiving, by at least one computing device from a first neural network, a set of 2D coordinates corresponding to the body features of the subject, wherein the first neural network is trained to use an entire image or image region as input and to return a respective set of 2D coordinates;
• detecting, by the at least one computing device using the received set of 2D coordinates, 2D Keypoints corresponding to the body features of the subject;
• transmitting, by the at least one computing device, to a second neural network, the detected 2D Keypoints, wherein the second neural network is trained to use the detected 2D Keypoints as input and to return corresponding depth estimation values;
• receiving, by the at least one computing device from the second neural network, the depth estimation values corresponding to the transmitted 2D Keypoints;
• processing, by the at least one computing device, the received depth estimation values to generate a set of 3D coordinates that correspond to the body features of the subject in camera space;
• applying, by the at least one computing device, one or more non-linear optimizations to fit a predetermined skeleton to the generate 3D Keypoints;
• and
• producing, by the at least one computing device, the pose estimation corresponding to the body features of the subject.