Markerless motion capture of hands with multiple pose estimation engines

A non-transitory medium with instructions stored thereon that, when executed by a processor, cause the processor to perform operations comprising:

• generating a first skeleton by applying, to an image of an individual, a first neural network that infers positions of a first plurality of joints in the image;
• identifying a portion of the image that includes a given anatomical region based on an analysis of the first skeleton;
• generating a second skeleton by applying, to at least the portion of the image, a second neural network that infers positions of a second plurality of joints in the portion of the image;
• and
• attaching the second skeleton to the first skeleton, so as to create a third skeleton that has more joints in the given anatomical region than the first skeleton.

An apparatus comprising:

• a camera that is configured to capture an image of an individual;
• a memory in which is stored
• (i) a first machine learning model that is trained to predict locations of a first plurality of joints across a human body, and
• (ii) a second machine learning model that is trained to predict locations of a second plurality of joints across an anatomical region of the human body;
• and
• a processor that is configured to:
• apply, to the image, the first machine learning model to generate predicted locations of the first plurality of joints in the image,
• apply, to at least a portion of the image that includes the anatomical region, the second machine learning model to generate predicted locations of the second plurality of joints in the image, and
• generate a skeleton with a higher density of joints in the anatomical region using the predicted locations of the first plurality of joints and the predicted locations of the second plurality of joints.