US 8,139,067 B2 · Shape completion, animation and marker-less motion capture of people, animals or characters

Abstract

Motion capture animation, shape completion and markerless motion capture methods are provided. A pose deformation space model encoding variability in pose is learnt from a three-dimensional (3D) dataset. Body shape deformation space model encoding variability in pose and shape is learnt from another 3D dataset. The learnt pose model is combined with the learnt body shape model. For motion capture animation, given parameter set, the combined model generates a 3D shape surface of a body in a pose and shape. For shape completion, given partial surface of a body defined as 3D points, the combined model generates a 3D surface model in the combined spaces that fits the 3D points. For markerless motion capture, given 3D information of a body, the combined model traces the movement of the body using the combined spaces that fits the 3D information or reconstructing the body's shape or deformations that fits the 3D information.

Claims35

Dependent claims

A shape completion method, comprising:

(a) using a processing pipeline to learn a pose deformation space model for a template mesh from a first three-dimensional example data set of pose mesh instances, said learnt pose deformation space model encoding variability in said template mesh with pose;
(b) using said processing pipeline to learn a body shape deformation space model for a template mesh from a second three-dimensional example data set of body shape mesh instances, said learnt body shape deformation space model encoding variability in said template mesh with body shape;
(c) using said processing pipeline to generate a partial mesh using the learnt pose deformation space model and said learnt body shape deformation space model based upon a provided pose and a provided body shape description;
and
(d) using said processing pipeline to complete said partial by generating a complete three-dimensional surface that fits said partial mesh.

2 more elements

A method of generating a mesh for a posed 3D character skeleton using a pose model generated from a set of pose mesh instances registered with respect to a template mesh and a decoupled body shape model generated from a set of body shape mesh instances registered with respect to the template mesh, wherein said template mesh is defined as a set of polygons, said template mesh has an associated articulated object skeleton and said body shape model includes a set of principal component analysis (PCA) parameters that define localized transformations applied to edges of polygons in said template mesh based upon a set of body shape parameters, comprising:

(a) providing to a processing pipeline said pose model, said body shape model, said template mesh and said articulated object skeleton;
(b) providing to said processing pipeline a pose for said articulated object skeleton;
(c) rotating rigid parts of said articulated object skeleton based upon said provided pose;
(d) using said processing pipeline to automatically synthesize a mesh for said pose by applying localized transformations to polygons in said template mesh based upon said joint angle rotations of said articulated object skeleton in said provided pose, wherein each of said localized transformations is applied to said edges of a polygon in said template mesh and is a function of said joint angle rotations of said joints in said articulated object skeleton closest to said polygon;
(e) providing to said processing pipeline a body shape description, wherein said body shape description comprises information indicative of a set of body shape parameters;
(f) using said processing pipeline to automatically deform said synthesized mesh by applying localized transformations to polygons in said synthesized mesh based upon the transformations defined by said PCA parameters defined in said body shape model and said set of body shape parameters;
and
(g) said processing pipeline outputting said deformed mesh, wherein said outputted mesh includes representations of muscular deformations associated with said provided pose.

5 more elements

Dependencies are inferred from the claim text.

Shape completion, animation and marker-less motion capture of people, animals or characters

Life of this patent

Shape completion, animation and marker-less motion capture of people, animals or characters

Life of this patent

Abstract

Claims35