Machine based sign language interpreter

A computer storage device including instructions for programming a processing device to perform a series of steps, comprising:

• capturing a scene using a capture device, the scene including a human target;
• tracking movements of a human target in the scene;
• detecting a plurality of gestures of the human target in the scene;
• assigning a first sign to a detected gesture by assigning a probability weight indicating strength of the match between the detected gesture and the first sign;
• assigning a second sign to an adjacent detected gesture by assigning a probability weight indicating the strength of the match between the adjacent detected gesture and the second sign;
• acquiring individual profiles of user motion for human targets applying known tendencies to motion and gesture detection for each human target;
• comparing the first sign and the second sign to verify accuracy of the second sign, including applying profile information of a detected human target to adjust a probability weight assigned for each sign;
• and
• generating an output to a display device indicating a written language visual translation of the first and second sign on the display device.

An image capture system including:

• a capture device including an RGB sensor and a depth sensor;
• a host device, the host device providing a video output, the host device including a processor and instructions for programming the processor to perform a method comprising:
• tracking movements of a human target in a scene acquired by the capture device,
• detecting one or more gestures of the human target in the scene;
• comparing the one or more gestures to a library of sign language signs and assigning a first sign to a first detected gesture, assigning a second sign to a second detected gesture, and assigning a third sign after the second sign to a third detected gesture, each said assigning including assigning a probability weight indicating strength of a match between the gesture and each said sign,
• comparing the first sign and the third sign to the second sign to determine the accuracy of the second sign to the first and third signs, said comparing including assigning a probability weight to each of the first and third signs indicating a strength of the match between the gesture and the sign based on a comparison to the second sign;
• acquiring individual profiles of user motion for users applying known tendencies to motion and gesture detection for a user to increase accuracy of gesture detection and sign language translation, each said comparing step using information from a human target profile in assigning a probability weight;
• and
• generating an output comprising a written language display of a meaning of the first, second and third signs based on the comparing steps.