System and method for gesture based data and command input via a wearable device

A data input system that includes a wearable device mounted on a wrist of a user and at least one computer device, wherein the wearable device provides gesture based data input to the computer device, the wearable device comprising:

• at least one primary camera configured to capture images of a hand connected with the wrist on which the wearable device is mounted, and at least one first object in physical environment of the wearable device, in order to track movements of the hand and the wrist;
• at least one inertia measurement unit comprising a gyroscope sensor and an accelerometer, wherein the at least one inertia measurement unit is configured to measure inertia information, which includes orientation, linear acceleration, and angular acceleration of the wrist on which the wearable device is mounted;
• at least one wired or wireless communication interface for communicating with the computer device;
• at least one or more secondary cameras mounted on a strap of the wearable device configured to capture images of at least one second object in physical environment of the wearable device, which is used for tracking movements of the wrist relative to the second object, wherein the second object may not be in a field of view of the primary camera;
• and
• a processor configured to:
• determine position of the wrist in 3D space, a first relative position to an initial position of the wrist, and orientation of the wrist in 3D space, by processing the inertia information;
• determine a relative orientation of the first object, with respect to the wrist, based on an analysis of images captured by the primary camera;
• determine position of the wrist in 3D space, a second relative position to the first object, by processing the images captured by the primary camera;
• and
• determine positions in 3D space of a fingertip, a PIP joint, and a DIP joint of at least one finger of the hand, a third relative position to at least one of the wearable device and a fourth relative position of the first object, by processing the images captured by the primary camera.