Method and apparatus of remote management of computer system using voice and gesture based input

A method of remote management of a computer system using voice and gesture based input, comprising:

• generating, by an application module of a management application executed at a computing device, a first user interface of the management application, wherein the first user interface shows options that allow a user to select at least one of a voice input and a gesture input for a baseboard management controller (BMC) communicatively connected to the computer system, wherein the computing device is remotely communicatively connected to the BMC through a network, and the BMC is configured to manage the computer system;
• displaying, by the application module on the computing device, the first user interface allowing the user to enter a first input, the first input including at least one of the voice input and the gesture input, wherein the computing device has an audio input device configured to receive the voice input, and to convert the voice input to audio input data, and a multitouch device configured to receive the gesture input, to detect multiple touch points simultaneously based on the gesture input, and to convert the detected multiple touch points to multitouch input data;
• receiving, by the application module, input data from the computing device representing the first input, the input data including at least one of the audio input data and the multitouch input data;
• determining, by the application module based on the input data, a first command from a plurality of commands targeted to the BMC managing the computer system;
• constructing, by the application module, a first message including the determined first command;
• and
• sending, by the application module, the first message from the computing device to the BMC over the network,
• wherein the BMC is configured to receive the first message from the computing device, and to perform the first command to manage the computer system based on the received first message.

A non-transitory computer storage medium having computer-executable instructions stored thereon which, when executed by a processor of a computing device, cause the processor to:

• generate a first user interface showing options that allow a user to select at least one of a voice input and a gesture input for a baseboard management controller (BMC) communicatively connected to a computer system, wherein the computing device is remotely communicatively connected to the BMC through a network, and the BMC is configured to manage the computer system;
• display the first user interface on the computing device for the user to enter a first input, the first input including at least one of the voice input and the gesture input, wherein the computing device has an audio input device configured to receive the voice input, and to convert the voice input to audio input data, and a multitouch device configured to receive the gesture input, to detect multiple touch points simultaneously based on the gesture input, and to convert the detected multiple touch points to multitouch input data;
• receive input data from the computing device representing the first input, the input data including at least one of the audio input data and the multitouch input data;
• determine, based on the input data, a first command from a plurality of commands targeted to the BMC managing the computer system;
• construct a first message including the determined first command;
• and
• send the first message from the computing device to the BMC over the network,
• wherein the BMC is configured to receive the first message from the computing device, and to perform the first command to manage the computer system based on the received first message.