System and method of supporting adaptive misrecognition in conversational speech

A method for processing natural language utterances, comprising:

• receiving a natural language utterance at a multimodal device;
• subsequently receiving a follow-up multimodal input at the multimodal device;
• recognizing one or more words from the natural language utterance using a speech recognition engine coupled to the multimodal device;
• generating an interpretation of the natural language utterance from the one or more recognized words using a parser coupled to the multimodal device, wherein the parser further generates a request based on the interpretation of the natural language utterance;
• invoking a domain agent configured to process the generated request;
• monitoring one or more actions associated with the domain agent processing the request using an adaptive misrecognition engine;
• and
• determining that the interpretation of the natural language utterance was incorrect if one or more of the monitored actions include the follow-up multimodal input being presented proximate in time to the prior natural language utterance.

A system for processing natural language utterances, comprising:

• a multimodal device configured to receive a natural language utterance;
• a speech recognition engine configured to recognize one or more words from the natural language utterance;
• a parser configured to generate an interpretation of the natural language utterance from the one or more recognized words, and further configured to generate a request based on the interpretation of the natural language utterance;
• a domain agent configured to process the generated request;
• an adaptive misrecognition engine configured to:
• monitor one or more actions associated with the domain agent processing the request;
• determine whether the interpretation of the natural language utterance is correct or incorrect based on the one or more monitored actions;
• and
• generate an unrecognized event if the interpretation of the natural language utterance is determined to be incorrect;
• and
• an analyzer configured to:
• analyze the unrecognized event to determine a frequency of incorrect interpretations for the request;
• and
• determine one or more tuning parameters for at least one of the speech recognition engine or the parser based on the frequency of incorrect interpretations for the request, wherein the tuning parameters are used to improve interpretations of subsequent natural language utterances relating to the request.

A system for processing natural language utterances, comprising:

• a multimodal device configured to receive a natural language utterance;
• a speech recognition engine configured to recognize one or more words from the natural language utterance;
• a parser configured to generate an interpretation of the natural language utterance from the one or more recognized words, and further configured to generate a request based on the interpretation of the natural language utterance;
• a domain agent configured to process the generated request;
• and
• an adaptive misrecognition engine configured to:
• monitor one or more actions associated with the domain agent processing the request;
• and
• determine that the interpretation of the natural language utterance was incorrect if one or more of the monitored actions include a user overriding the request in a time that is shorter than an expected time for processing the request.

A method for processing natural language utterances, comprising:

• receiving a natural language utterance at a multimodal device;
• recognizing one or more words from the natural language utterance using a speech recognition engine coupled to the multimodal device;
• generating an interpretation of the natural language utterance from the one or more recognized words using a parser coupled to the multimodal device, wherein the parser further generates a request based on the interpretation of the natural language utterance;
• invoking a domain agent configured to process the generated request;
• monitoring one or more actions associated with the domain agent processing the request using an adaptive misrecognition engine;
• determining whether the interpretation of the natural language utterance is correct or incorrect based on the actions monitored using the adaptive misrecognition engine;
• generating an unrecognized event if the adaptive misrecognition engine determines that the interpretation of the natural language utterance is incorrect;
• analyzing the unrecognized event to determine a frequency of incorrect interpretations for the request;
• and
• determining one or more tuning parameters for at least one of the speech recognition engine or the parser based on the frequency of incorrect interpretations for the request, wherein the tuning parameters are used to improve interpretations of subsequent natural language utterances relating to the request.

A method for processing natural language utterances, comprising:

• receiving a natural language utterance at a multimodal device;
• recognizing one or more words from the natural language utterance using a speech recognition engine coupled to the multimodal device;
• generating an interpretation of the natural language utterance from the one or more recognized words using a parser coupled to the multimodal device, wherein the parser further generates a request based on the interpretation of the natural language utterance;
• invoking a domain agent configured to process the generated request;
• monitoring one or more actions associated with the domain agent processing the request using an adaptive misrecognition engine;
• and
• determining that the interpretation of the natural language utterance was incorrect if one or more of the monitored actions include a user overriding the request in a time that is shorter than an expected time for processing the request.