Approaches to estimating hand pose with independent detection of hand presence in digital images of individuals performing physical activities and systems for implementing the same

A method performed by a computer program executed on a computing device, the method comprising:

• receiving multiple digital images;
• determining locations of one or more anatomical landmarks in each of the multiple digital images;
• determining, based on the locations, spatial positions of one or more hands in each of the multiple digital images;
• for each hand determined to be in one of the multiple digital images,
• placing a bounding box around the hand, and
• iteratively displacing the bounding box until the bounding box does not include the spatial positions of the one or more hands;
• for each displaced bounding box, adding a portion of the multiple digital images that is associated with the displaced bounding box to a training dataset;
• training a first branch of a neural network on the training dataset;
• receiving a digital image of a scene that includes an individual performing a physical activity in an environment;
• segmenting the digital image into multiple segments, each of which is representative of a contiguous region of pixels that is associated with a portion of the scene;
• for each of the multiple segments, extracting a feature map so as to produce multiple feature maps, each of which is a vectorial representation of content in a corresponding one of the multiple segments;
• and
• for each of the multiple feature maps,
• applying the neural network so as to:
• determine, via the first branch of the neural network, a likelihood that the corresponding one of the multiple segments includes a hand, and
• determine, via a second branch of the neural network, an estimated pose of the hand in the corresponding one of the multiple segments;
• comparing the likelihood to a threshold value programmed in memory of the computing device;
• and
• responsive to a determination that the likelihood exceeds the threshold value, storing an indication that the hand is in the estimated pose.