Monocular tracking of 3D human motion with a coordinated mixture of factor analyzers

A system for tracking three-dimensional (3D) motion of a subject comprising:

• image receiving means for receiving a two-dimensional (2D) image sequence;
• model receiving means for receiving a prediction model learned from offline learning means, the prediction model specifying a mapping of pose information of the subject between a high dimensional pose space and a low dimensional pose space, the high dimensional pose space having a higher dimensionality than the low dimensional pose space;
• pose prediction means for generating, in the low dimensional pose space, a predicted pose state of the subject based on the prediction model;
• and
• tracking means for generating 3D tracking data to track the motion in the high dimensional pose space based at least in part on the predicted pose state and the 2D image sequence;
• wherein the offline learning means comprises:
• 3D capture means for capturing 3D motion data;
• processing means for processing the 3D motion data to extract training pose information;
• model learning means for learning a dimensionality reduction model based on the extracted training pose information adapted to bidirectionally map the pose training information from the high dimensional pose space to the low dimensional pose space;
• and
• prediction model generating means for generating the prediction model based at least in part on the dimensionality reduction model.

A computer program product, comprising a computer readable medium storing computer executable code for tracking three-dimensional (3D) motion of a subject, the computer executable code when executed causing a computer to perform the steps of:

• receiving a two-dimensional (2D) image sequence;
• receiving a prediction model learned in an offline learning process, the prediction model specifying a mapping of pose information of the subject between a high dimensional pose space and a low dimensional pose space, the high dimensional pose space having a higher dimensionality than the low dimensional pose space;
• generating, in the low dimensional pose space, a predicted pose state of the subject based on the prediction model;
• and
• generating 3D tracking data to track the motion in the high dimensional pose space based at least in part on the predicted pose state and the 2D image sequence;
• wherein the offline learning process comprises steps of:
• capturing 3D motion data;
• processing the 3D motion data to extract training pose information;
• learning a dimensionality reduction model based on the extracted training pose information adapted to bidirectionally map the pose training information from the high dimensional pose space to the low dimensional pose space;
• and
• generating the prediction model based at least in part on the dimensionality reduction model.

A method for tracking three-dimensional (3D) motion of a subject comprising the steps of:

• receiving a two-dimensional (2D) image sequence;
• receiving a prediction model learned in an offline learning process, the prediction model specifying a mapping of pose information of the subject between a high dimensional pose space and a low dimensional pose space, the high
• dimensional pose space having a higher dimensionality than the low dimensional pose space;
• generating, in the low dimensional pose space, a predicted pose state of the subject based on the prediction model;
• and
• generating 3D tracking data to track the motion in the high dimensional pose space based at least in part on the predicted pose state and the 2D image sequence;
• wherein generating 3D tracking data comprises:
• generating a prior probability density function based on the prediction model and at least one previous 2D image in the image sequence;
• generating a likelihood function based on a matching cost between the predicted pose state and the current 2D image;
• and
• generating a posterior probability density function based on the prior probability density function and the likelihood function.

A computer program product comprising a computer readable medium storing computer executable code for tracking three-dimensional (3D) motion of a subject, the computer executable code when executed causing a computer to perform the steps of:

• receiving a two-dimensional (2D) image sequence;
• receiving a prediction model learned in an offline learning process, the prediction model specifying a mapping of pose information of the subject between a high dimensional pose space and a low dimensional pose space, the high dimensional pose space having a higher dimensionality than the low dimensional pose space,
• generating, in the low dimensional pose space, a predicted pose state of the subject based on the prediction model;
• and
• generating 3D tracking data to track the motion in the high dimensional pose space based at least in part on the predicted pose state and the 2D image sequence;
• wherein generating 3D tracking data comprises:
• generating a prior probability density function based on the prediction model and at least one previous 2D image in the image sequence;
• generating a likelihood function based on a matching cost between the predicted pose state and the current 2D image;
• and
• generating a posterior probability density function based on the prior probability density function and the likelihood function.

A method for learning a prediction model for three-dimensional (3D) human motion tracking, the method comprising the steps of:

• capturing 3D motion data;
• processing the 3D motion data to extract pose information;
• learning a dimensionality reduction model based on the extracted pose information adapted to bidirectionally map the pose information from the high dimensional space to the low dimensional space;
• and
• generating a prediction model based at least in part on the dimensionality reduction model.

A computer program product, comprising a computer readable medium storing computer executable code for learning a prediction model for three-dimensional human motion tracking, the computer executable code when executed causing a computer to perform the steps of:

• capturing 3D motion data;
• processing the 3D motion data to extract pose information;
• learning a dimensionality reduction model based on the extracted pose information adapted to bidirectionally map the pose information from the high dimensional space to the low dimensional space;
• and
• generating a prediction model based at least in part on the dimensionality reduction model.