System for fast, probabilistic skeletal tracking

A software pipeline for generating a state estimate for a given frame of captured image data, the state estimate representing an estimate of a position of a user within a field of view captured within the image data, comprising:

• a preprocessing routine for receiving the image data, removing a background from the image data, and processing a foreground into one or more body part proposals;
• one or more experts for receiving information including the one or more body part proposals and generating a plurality of computer models, each computer model representing an estimation of the position of the user in the given frame of captured image data;
• and
• an arbiter for receiving the plurality of computer models, scoring the computer models by one or more methodologies which compare the plurality of computer models against depth data from the given frame and/or state estimate data from a prior frame, and outputting at least one computer model estimated by the arbiter to best approximate the position of the user in the frame.

A computer-readable storage medium capable of programming a processor to perform a method tracking body parts of a user captured with a capture device as a user moves within a field of view of the capture device to determine a state estimate of the user's position in a current frame of image data captured by the capture device, comprising:

• (a) receiving image data from the capture device;
• (b) processing the image data received in said step (a) to remove a background and generate body part proposals for images in a foreground;
• (c) generating a plurality of skeletal hypotheses using at least one of the body part proposals generated in said step (b) and image data from an earlier time, the plurality of skeletal hypotheses defining a probability distribution;
• (d) selecting one or more skeletal hypotheses as being the most probable state estimates based on the probability distribution, where the probability distribution indicates one or more skeletal hypotheses as the probable state estimate;
• and
• (e) indicating that no state estimate is determined for the frame of image data where the probability distribution does not indicate one or more skeletal hypotheses as being probable state estimates.