Monocular Hand Tracking

A system comprising:

• a non-transitory computer-readable storage medium;
• and
• one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the one or more processors to perform operations comprising:
• obtaining live data comprising an image from a camera of the electronic device, the image depicting a hand of a user at a point in time:
• obtaining enrollment data corresponding to a size of at least a portion of a hand, wherein the enrollment data is based on sensor data of the hand of the user obtained prior to the live data being obtained;
• determining parameters for a hand representation based on the live data and the enrollment data, wherein the parameters comprise rotation parameter values corresponding to hand joints and hand pose parameter values corresponding to hand pose;
• and
• determining a three-dimensional (3D) position and configuration of the hand based on the parameters of the hand representation.

A non-transitory computer-readable storage medium storing program instructions executable via one or more processors to perform operations comprising:

• obtaining live data comprising an image from a camera of the electronic device, the image depicting a hand of a user at a point in time;
• obtaining enrollment data corresponding to a size of at least a portion of a hand, wherein the enrollment data is based on sensor data of the hand of the user obtained prior to the live data being obtained;
• determining parameters for a hand representation based on the live data and the enrollment data, wherein the parameters comprise rotation parameter values corresponding to hand joints and hand pose parameter values corresponding to hand pose;
• and
• determining a three-dimensional (3D) position and configuration of the hand based on the parameters of the hand representation.