Natural language grammar improvement

A computer-implemented method for configuring natural language grammars, the method comprising:

• for a stored audio query,
• generating, using a processor, a first transcription of the audio query, the first transcription having (i) first automatic speech recognition (ASR) score and (ii) a first natural language understanding (NLU) score, the first NLU score resulting from a first weighted grammar associated with the first transcription;
• and
• identifying a second transcription of the audio query, the second transcription having (i) a second ASR score and (ii) a second NLU score;
• detecting that a difference between the first ASR score and the second ASR score has signed value with an opposite sign than a sign of a signed value of a difference between the first NLU score and the second NLU score;
• and
• responsive to detecting the opposite sign:
• providing, to an evaluator, the audio query and the first and second transcriptions;
• receiving, from the evaluator, an indication of which of the first transcription and the second transcription is a correct transcription;
• adjusting, using a processor, a value implemented to calculate the first NLU score or a value implemented to calculate the second NLU score;
• and
• calculating at least one of the first NLU score and the second NLU score using the adjusted value.