Rule based application execution using multi-modal inputs

An application executing method performed in an electronic device, the method comprising:

• executing a first application and a second application in a first mode corresponding to a mode for recognizing only an utterance input of a user;
• recognizing a specified user input for entering a second mode corresponding to a mode for recognizing the utterance input and an auxiliary input of the user together;
• operating the first application and the second application in the second mode based on the recognized specified user input;
• receiving an utterance input based on a user voice of the user received via a microphone in the second mode in which the second application is operated in a background state;
• identifying a first unspecified parameter corresponding to a part of the utterance input received at a first time, among a plurality of parameters for generating a rule corresponding to the utterance input;
• receiving a first auxiliary input via a display or an external input device, within a specified time interval from the first time;
• identifying at least one first item based on the first auxiliary input;
• determining a parameter in place of the first unspecified parameter based on the at least one first item;
• identifying the rule associated with at least one function of the first application based on the utterance input and the determined parameter;
• operating the at least one function of the first application based on the identified rule;
• and
• changing the first application and the second application from the second mode to the first mode based on not receiving an auxiliary input from the user for a specified time.

A non-transitory recording medium recording instructions of a method executable by a processor of an electronic device, wherein the instructions cause the processor to perform:

• executing a first application and a second application in a first mode corresponding to a mode for recognizing only an utterance input of a user;
• recognizing a specified user input for entering a second mode corresponding to a mode for recognizing the utterance input and an auxiliary input of the user together;
• operating the first application and the second application in the second mode based on the recognized specified user input;
• receiving an utterance input based on a user voice of the user received via a microphone of the electronic device in the second mode in which the second application is operated in a background state;
• identifying a unspecified parameter corresponding to a part of the utterance input received at a first time, among a plurality of parameters for generating a rule corresponding to the utterance input;
• receiving a first auxiliary input via a display or an external input device, within a specified time interval from the first time;
• determining at least one first item based on the first auxiliary input;
• determining a parameter in place of the unspecified parameter based on the at least one first item;
• identifying the rule associated with at least one function of the first application based on the utterance input and the determined parameter;
• operating the at least one function of the first application based on the identified rule;
• and
• changing the first application and the second application from the second mode to the first mode based on not receiving an auxiliary input from the user for a specified time.