Multimodal interaction using a state machine and hand gestures discrete values

A computer implemented method for associating between a computerized model of multimodal human interaction and at least one application function, comprising:

• accessing a memory configured to store a plurality of hand gestures in a three dimensional (3D) space, each one of said plurality hand gestures is defined by a dataset defining a spatial orientation of a hand of a human by at least one of a plurality of discrete pose values and at least one of a plurality of discrete motion values;
• receiving instructions from a programmer to create for an application having an application code, using at least one processor, a logical sequence of user input that combines at least one of said plurality of hand gestures with at least one non-gesture user input;
• and
• generating, using said at least one processor, a multimodal act code for said application, said multimodal act code associates said logical sequence with at least one application function per instructions of said programmer;
• adding said multimodal act code to said application code at a built time prior to execution of said application code by a target computerized device;
• wherein an execution of said multimodal act code by said target computerized device triggers said at least one application function in response to a detection of said logical sequence from an analysis of data captured using at least one capturing unit of said target computerized device, said data documents user interactions during runtime of said application.

A software program product for associating between a computerized model of multimodal human interaction and at least one application function, comprising:

• a non-transitory computer readable storage medium;
• first program instructions to access a storage configured to store a plurality of hand gestures in a three dimensional (3D) space, each one of said plurality hand gestures is defined by a dataset defining a spatial orientation of a hand of a human by at least one of a plurality of discrete pose values and at least one of a plurality of discrete motion values;
• second program instructions to receive instructions from a programmer to create an application having an application code, a logical sequence of user input that combines at least one of said plurality of hand gestures with at least one non-gesture user input;
• and
• third program instructions to associate generate a multimodal act code for said application, said multimodal act code associates said logical sequence with at least one application function per instructions of said programmer;
• fourth program instructions to add said multimodal act code to said application code at a built time prior to execution of said application code by a target computerized device;
• wherein an execution of said multimodal act code by said target computerized device triggers said at least one application function in response to a detection of said logical sequence from an analysis of data captured using at least one capturing unit of said target computerized device, said data documents user interactions during runtime of said application;
• wherein said first, second, third, and fourth program instructions are executed by at least one computerized processor from said non-transitory computer readable storage medium.