US 11,380,121 B2 · Full skeletal 3D pose recovery from monocular camera

Abstract

Recent progress of neural networks has shown significant progress over human pose estimation tasks. Pose estimation could be categorized into monocular 2D pose estimation, multi-view 3D pose estimation, and single view 3D pose estimation, where recently 3D pose is getting more attention to be applied to AR/VR, game and human computer interaction applications. However, current academic benchmarks on human 3D pose estimation only respect performance on their relative pose. The root positioning over the time, in another word, the “trajectory” of the whole body in 3D space is not considered well enough. Applications such as motion capture, not only require precise relative pose of the body but also the root position of the whole body in 3D space. Therefore, an efficient monocular full 3D pose recovery model from 2D pose input is described herein, which is able to be applied to the above applications.

Claims21

Dependent claims

A method comprising:

receiving camera information, wherein the camera information includes a 2-dimensional pose and camera parameters including a focal length;
applying feature extraction on the camera information, including residual determination with 1-dimensional convolution;
estimating bone lengths based on the feature extraction;
estimating bone unit vectors based on the feature extraction, conditioned on the bone lengths;
and
estimating relative poses from the bone lengths and the bone unit vectors, and deriving root positions based on the feature extraction, conditioned on the bone lengths and the bone unit vectors.

3 more elements

Dependencies are inferred from the claim text.

Full skeletal 3D pose recovery from monocular camera

Life of this patent

Full skeletal 3D pose recovery from monocular camera

Life of this patent

Abstract

Claims21