Determining voice recognition accuracy in a voice recognition system

A method for determining voice recognition accuracy of a voice recognition system that recognizes a voice command in a user utterance, comprising:

• analyzing voice recognition information produced by a voice recognition system, including comparing a manually transcribed version of the user utterance to the recognized voice command, comprising analyzing a response time for the voice recognition system to recognize the user utterance as the recognized voice command;
• detecting, in a computing device, at least one recognition error based on the analysis;
• determining at least one source of error for the detected recognition error;
• implementing at least one solution to eliminate the source of error;
• and
• reducing a perplexity of the grammar if the response time exceeds a threshold value, the perplexity based on a quantity of decision levels and a quantity of voice commands.

A method for determining voice recognition accuracy of a voice recognition system that recognizes a voice command in a user utterance, comprising:

• analyzing voice recognition information produced by a voice recognition system, including comparing a manually transcribed version of the user utterance to the recognized voice command, comprising analyzing a quantity of timeout errors associated with a timeout period, the timeout period defining a period of time for a user to provide the user utterance;
• detecting, in a computing device, at least one recognition error based on the analysis;
• determining at least one source of error for the detected recognition error;
• implementing at least one solution to eliminate the source of error;
• and
• increasing the timeout period if the quantity of timeout errors exceeds a threshold value.

A tangible computer-readable medium having computer program logic stored thereon that enables a computing device to provide an automated method for determining voice recognition accuracy of a voice recognition system that recognizes a voice command in a user utterance, the computer program logic comprising:

• an analyzing module configured to analyze voice recognition information produced by the voice recognition system and to compare a manually transcribed version of the user utterance to the recognized voice command;
• a detecting module configured to detect at least one recognition error based on the analysis, comprising detecting at least one of:
• an in-grammar false accept (IGFA) type error,
• an in-grammar false reject (IGFR) type error,
• an out-of-grammar false accept (OGFA) type error, or
• an out-of-grammar correct reject (OGCR) type error;
• a determining module configured to determine at least one source of error for the detected recognition error;
• and
• a generating module configured to generate at least one solution to eliminate the source of error.

A system for determining voice recognition accuracy of a voice recognition system that recognizes a voice command in a user utterance, comprising:

• an analysis module that analyzes voice recognition information produced by the voice recognition system and comparing a manually transcribed version of the user utterance to the recognized voice command;
• an error detection module that detects at least one recognition error based on the analysis, comprising detecting at least one of:
• an in-grammar false accept (IGFA) type error,
• an in-grammar false reject (IGFR) type error,
• an out-of-grammar false accept (OGFA) type error, or
• an out-of-grammar correct reject (OGCR) type error;
• an error source identification module that identifies at least one source of error for the detected recognition error;
• and
• a solution generation module that generates at least one solution to eliminate the source of error,
• wherein at least one of the analysis module, the error detection module, the error source identification module, or the solution generation module is executed by a computing device.

A device for determining voice recognition accuracy of a voice recognition system that recognizes a voice command in a user utterance, comprising:

• an analysis module that analyzes voice recognition information produced by the voice recognition system and compares a manually transcribed version of the user utterance to the recognized voice command;
• an error detection module that detects at least one recognition error based on the analysis, comprising detecting at least one of:
• an in-grammar false accept (IGFA) type error,
• an in-grammar false reject (IGFR) type error,
• an out-of-grammar false accept (OGFA) type error, or
• an out-of-grammar correct reject (OGCR) type error;
• an error source identification module that identifies at least one source of error for the detected recognition error;
• and
• a solution generation module that generates at least one solution to eliminate the source of error.

A computer system for determining voice recognition accuracy of a voice recognition system that recognizes a voice command in a user utterance, comprising:

• an interface that receives voice recognition information produced by a voice recognition system;
• a memory;
• and
• at least one processor that executes computer program logic residing in the memory, wherein the at least one processor executes:
• analysis logic to analyze the voice recognition information and to compare a manually transcribed version of the user utterance to the recognized voice command,
• error detection logic to detect at least one recognition error based on the analysis, comprising detecting at least one of:
• an in-grammar false accept (IGFA) type error,
• an in-grammar false reject (IGFR) type error,
• an out-of-grammar false accept (OGFA) type error, or
• an out-of-grammar correct reject (OGCR) type error,
• error source identification logic to identify at least one source of error for the detected recognition error, and
• solution generation logic to generate at least one solution to eliminate the source of error.