Single-camera computer vision system for three-dimensional biomechanical motion reconstruction

A method for generating a three-dimensional body reconstruction from monocular video, comprising:

• receiving, by one or more processors, a sequence of image frames depicting a human subject captured by a single camera;
• extracting, by the one or more processors, image features from the sequence of image frames;
• processing the image features through a first neural network model to regress three-dimensional body mesh vertices;
• processing the image features through a second neural network model, distinct from the first neural network model, to produce body pose parameters in a camera-space reference frame;
• geometrically aligning, by the one or more processors, outputs of the first neural network model and the second neural network model using a rigid transformation to produce an aligned three-dimensional body reconstruction;
• and
• generating at least one biomechanical metric from the aligned three-dimensional body reconstruction.

A system for generating a three-dimensional body reconstruction from monocular video, comprising:

• a non-transitory memory storing instructions;
• and
• one or more processors coupled to the non-transitory memory and configured to execute the instructions to:
• receive a sequence of image frames depicting a human subject captured by a single camera;
• extract image features from the sequence of image frames;
• process the image features through a first neural network model to regress three-dimensional body mesh vertices;
• process the image features through a second neural network model, distinct from the first neural network model, to produce body pose parameters in a camera-space reference frame;
• geometrically align outputs of the first neural network model and the second neural network model using a rigid transformation to produce an aligned three-dimensional body reconstruction;
• and
• generate at least one biomechanical metric from the aligned three-dimensional body reconstruction.

A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

• receiving a sequence of image frames depicting a human subject captured by a single camera;
• extracting image features from the sequence of image frames;
• processing the image features through a first neural network model to regress three-dimensional body mesh vertices;
• processing the image features through a second neural network model, distinct from the first neural network model, to produce body pose parameters in a camera-space reference frame;
• geometrically aligning outputs of the first neural network model and the second neural network model using a rigid transformation to produce an aligned three-dimensional body reconstruction;
• and
• generating at least one biomechanical metric from the aligned three-dimensional body reconstruction.