System and Method For Enhanced Gesture-Based Interaction

A gesture-based interactive computerised system comprising:—

• a three-dimensional imaging system;
• a gesture recognition system;
• a user-operated control device comprising:
• a housing;
• at least one sensing module including at least one control means mounted in the housing, each control means being capable of generating a control signal for the computerised system when operated;
• a feedback module mounted in the housing for providing contextual feedback to the user in accordance with an application operating on the computerised system;
• a communication module located within the housing and through which each control signal is transmitted to, and each feedback signal is received from, the computerised system;
• and a controller module mounted in the housing, the controller controlling the sensing module, the communication module and the feedback module in accordance with each generated control signal;
• and
• a computerised system associated with the three-dimensional imaging system and the gesture recognition system, the computerised system processing images captured by the three-dimensional imaging system and gestures recognised by the gesture recognition system;
• wherein the computerised system uses data from the user-operated control device in conjunction with the gestures recognised by the gesture recognition system to resolve ambiguities in information measured by the three-dimensional imaging system.

A non-transitory computer readable medium storing a program causing a computer to execute a method for controlling an interactive computerised system having a three-dimensional imaging system, the three-dimensional imaging system including a three-dimensional camera for providing three-dimensional information of at least one part of a user present within its frustum, the method comprising the steps of:

• a) providing a user-operated control device to be associated with at least one part of the user, the user-operated control device comprising:
• a housing;
• at least one sensing module including at least one control means mounted in the housing, each control button being capable of generating a control signal for the computerised system when operated;
• at least one feedback module mounted in the housing for providing contextual feedback to the user in accordance with an application operating on the computerised system;
• a communication module located within the housing and through which each control signal is transmitted to, and each feedback signal is received from, the computerised system;
• and a controller module mounted in the housing, the controller controlling the sensing module, the communication module and the feedback module in accordance with each generated control signal;
• b) generating control signals from at least one control means on the user-operated control device;
• and
• c) using the generated control signals to control the interactive computerised system;
• wherein the method further comprises the step of using the information provided by the user-operated control device relating to the part of the user with which it is associated to resolve ambiguities in information measured by the three-dimensional imaging system.