Multi-modal input control of touch-based devices

A method of multi-modal control of a touch-based device, the method comprising:

• receiving multi-modal control inputs from one or more of voice commands, a game controller, a handheld remote, and physical gestures detected by a sensor;
• converting the multi-modal control inputs into corresponding virtual inputs which model physical inputs recognizable by the touch-based device;
• and
• providing the corresponding virtual inputs to the touch-based device for control thereof, wherein the virtual inputs are utilized by the touch-based device as corresponding physical inputs without modifying underlying applications executed on the touch-based device for control with the multi-modal control inputs.

An apparatus for multi-modal control of a touch-based device, the method comprising:

• circuitry configured to receive multi-modal control inputs from one or more of voice commands, a game controller, a handheld remote, and physical gestures detected by a sensor;
• circuitry configured to convert the multi-modal control inputs into corresponding virtual inputs which model physical inputs recognizable by the touch-based device;
• and
• circuitry configured to provide the corresponding virtual inputs to the touch-based device for control thereof, wherein the virtual inputs are utilized by the touch-based device as corresponding physical inputs without modifying underlying applications executed on the touch-based device for control with the multi-modal control inputs.

A non-transitory computer-readable storage medium having computer readable code stored thereon for programming a computer, server, appliance, or device, each of which comprising a processor which, in response to the computer readable code, performs steps of

• receiving multi-modal control inputs from one or more of voice commands, a game controller, a handheld remote, and physical gestures detected by a sensor;
• converting the multi-modal control inputs into corresponding virtual inputs which model physical inputs recognizable by the touch-based device;
• and
• providing the corresponding virtual inputs to the touch-based device for control thereof, wherein the virtual inputs are utilized by the touch-based device as corresponding physical inputs without modifying underlying applications executed on the touch-based device for control with the multi-modal control inputs.