System and method for gesture recognition in three dimensions using stereo imaging and color vision

A method for recognizing gestures comprising:

• obtaining an image data;
• determining a hand pose estimation;
• producing a frontal view of a hand based on performing perspective unwarping to produce an unwarped frontal view of the hand and scaling the unwarped frontal view of the hand into a template image;
• isolating the hand from the background;
• and
• classifying a gesture of the hand.

A method for recognizing gestures comprising:

• obtaining an image data;
• determining a hand pose estimation;
• producing a frontal view of a hand;
• isolating the hand from the background;
• classifying a gesture of the hand;
• and
• matching the hand template against a plurality of gesture templates based on computing geometric moments of a first order, a second order and a third order;
• and applying a Mahalanobis distance metric.

A method for recognizing gestures comprising:

• obtaining an image data;
• performing background subtraction;
• computing a hand pose estimation based on an arm orientation determination;
• performing perspective unwarping to produce an unwarped frontal view of a hand;
• scaling the unwarped frontal view of the hand into a template image;
• isolating the hand from the background using color segmentation;
• and
• classifying a gesture of the hand by matching the hand with a plurality of template hand images.

A system comprising:

• a stereo camera coupled to a computer, the computer comprising a processor and a storage device to read from a machine readable medium, the machine readable medium containing instructions which, when executed by the processor, cause the computer to perform operations comprising:
• obtaining an image data;
• performing background subtraction;
• computing a hand pose estimation based on an arm orientation determination;
• performing perspective unwarping to produce an unwarped frontal view of a hand;
• scaling the unwarped frontal view of the hand into a template image;
• isolating the hand from the background using color segmentation;
• and
• classifying a gesture of the hand by matching the hand with a plurality of template hand images.

A method for recognizing gestures comprising:

• obtaining an image data;
• determining a hand pose estimation based on performing background subtraction, determining an arm orientation, and computing the hand pose estimation based on the arm orientation, wherein performing background subtraction includes selecting as a foreground arm image those pixels where the difference between a mean background depth and the current depth is larger than an empirically defined threshold;
• producing a frontal view of a hand;
• isolating the hand from the background;
• and
• classifying a gesture of the hand.

A method for recognizing gestures comprising

• obtaining an image data;
• determining a hand pose estimation based on performing background subtraction, determining an arm orientation, and computing the hand pose estimation based on the arm orientation, wherein determining an arm orientation includes fitting a first two dimensional (2D) line to the image data in the image plane;
• fitting a second 2D line to the image data in the depth dimension in the plane containing the first 2D line such that the second 2D line is perpendicular to the viewing plane, and combining the first 2D line and the second 2D line into a three-dimensional (3D) line such that the 3D line defines the arm orientation;
• producing a frontal view of a hand;
• isolating the hand from the background;
• and
• classifying a gesture of the hand.