US 10,997,787 B2 · 3D hand shape and pose estimation

Abstract

Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for receiving a monocular image that includes a depiction of a hand and extracting features of the monocular image using a plurality of machine learning techniques. The program and method further include modeling, based on the extracted features, a pose of the hand depicted in the monocular image by adjusting skeletal joint positions of a three-dimensional (3D) hand mesh using a trained graph convolutional neural network (CNN); modeling, based on the extracted features, a shape of the hand in the monocular image by adjusting blend shape values of the 3D hand mesh representing surface features of the hand depicted in the monocular image using the trained graph CNN; and generating, for display, the 3D hand mesh adjusted to model the pose and shape of the hand depicted in the monocular image.

Claims20

Dependent claims

A method comprising:

receiving, by one or more processors, a monocular image that includes a depiction of a hand;
modeling, by the one or more processors, a pose of the hand depicted in the monocular image by adjusting skeletal joint positions of a three-dimensional (3D) hand mesh using a trained graph convolutional neural network (CNN), the trained graph CNN estimating 3D coordinates of vertices in the 3D hand mesh;
linearly regressing the joint positions using a linear graph CNN;
and
generating, for display by the one or more processors, the 3D hand mesh adjusted to model the pose of the hand depicted in the monocular image.

2 more elements

Dependencies are inferred from the claim text.

3D hand shape and pose estimation

Life of this patent

3D hand shape and pose estimation

Life of this patent

Abstract

Claims20