# US 2019/0244611 A1 · System and method for automatic filtering of test utterance mismatches in automatic speech recognition systems

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0244611 A1
- Document ID: US20190244611A1
- Status: Published application
- Kind: A1
- Application number: 16/265,148
- Filed: 2019-02-01
- Published: 2019-08-08
- Earliest claimed: 2018-02-02
- Assignees: Genesys Telecommunications Laboratories, Inc.
- Applicants: Genesys Telecommunications Laboratories, Inc.
- Inventors: Tejas Godambe; Aravind Ganapathiraju
- Web page: https://patentgrep.com/patent/US20190244611A1

## CPC

- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 15/02: Feature extraction for speech recognition; Selection of recognition unit
- G10L 15/14: using statistical models, e.g. Hidden Markov Models [HMMs] (G10L15/18 takes precedence)
- G10L 15/16: using artificial neural networks
- G10L 15/197: Probabilistic grammars, e.g. word n-grams
- G10L 2015/025: {Phonemes, fenemes or fenones being the recognition units}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A system and method are presented for the automatic filtering of test utterance mismatches in automatic speech recognition (ASR) systems. Test data are evaluated for match between audio and text in a language-independent manner. Utterances having mismatch are identified and isolated for either removal or manual verification to prevent incorrect measurements of the ASR system performance. In an embodiment, contiguous stretches of low probabilities in every utterance are searched for and removed. Such segments may be intra-word or cross-word. In another embodiment, scores may be determined using log DNN probability for every word in each utterance. Words may be sorted in the order of the scores and those utterances containing the least word scores are removed.

## Claims

### Claim 1

A method for the automatic filtering of test utterance mismatches between a transcription and an audio signal in automatic speech recognition systems, the method comprising:
creating an alignment of the audio signal with the reference transcription;
determining Deep Neural Network probabilities for a senone sequence from the alignment;
identifying a contiguous stretch of senone probabilities that do not meet a threshold in each utterance;
locating a segment of near zero senone probabilities within the contiguous stretch; and
removing any of the utterances.

### Claim 2 (depends on 1)

The method of claim 1, wherein the test utterances comprise data containing longer utterances.

### Claim 3 (depends on 1)

The segment of claim 1, wherein the segment is of a predetermined length.

### Claim 4 (depends on 3)

The segment of claim 3, wherein the segment is intra-word.

### Claim 5 (depends on 3)

The segment of claim 3, wherein the segment is cross-word.

### Claim 6 (depends on 1)

The method of claim 1, wherein the automatic speech recognition system is language independent.

### Claim 7

A method for the automatic filtering of test utterance mismatches between a transcription and an audio signal in automatic speech recognition systems, the method comprising:
creating an alignment of the audio signal with the reference transcription;
determining a score using log Deep Neural Network probabilities for every word in each utterance of the audio signal;
sorting the words in order of their associated scores; and
removing the utterances containing problematic words that do not meet a threshold.

### Claim 8 (depends on 7)

Method of claim 7, wherein the determining of a score is computed as an average of log of senone probabilities over a length of the word.

### Claim 9 (depends on 8)

The method of claim 8, wherein the determining is performed using the mathematical formula:
S
w
=
1
m

∑
t
=
1
m

log


p
senone
t
,
where m represents a number of senones in a word.

### Claim 10 (depends on 7)

The method of claim 7, wherein the determining of a score is computed as an average of scores of all phones in a word.

### Claim 11 (depends on 10)

The method of claim 10, wherein the score is determined as an average of log senone probabilities over the length of a phone.

### Claim 12 (depends on 10)

The method of claim 10, wherein the determining is performed using the mathematical formula:
S
w
=
1
n

∑
ph
=
1
n

1
m
ph

∑
t
=
1
m
ph

log


p
senone
t
,

where n represents a number of phones in a word and mph represents a number of senones in a phone.

### Claim 13 (depends on 7)

The method of claim 7, wherein the determining of a score is computed as an average of scores of all phones in a word.

### Claim 14 (depends on 13)

The method of claim 13, wherein the determining is performed using the mathematical formula:
S
w
=
1
n

∑
ph
=
1
n

max
1
<
t
<
m
ph

(
log


p
senone
t
)
,
where n represents a number of phones in a word and mph represents a number of senones in a phone.

### Claim 15 (depends on 13)

The method of claim 13, where a phone score is determined as a maximum of log senone probabilities in a phone.

### Claim 16 (depends on 7)

The method of claim 7, wherein the automatic speech recognition system is language independent.

### Claim 17

A system for the automatic filtering of test utterance mismatches between a transcription and an audio signal in automatic speech recognition systems, the system comprising:
a processor; and
a memory in communication with the processor, the memory storing instructions that, when executed by the processor, causes the processor to:
create an alignment of the audio signal with the reference transcription,
determine Deep Neural Network probabilities for a senone sequence from the alignment,
identify a contiguous stretch of senone probabilities that do not meet a threshold in each utterance,
locate a segment of near zero senone probabilities within the contiguous stretch, and
remove any of the utterances.

### Claim 18

A system for the automatic filtering of test utterance mismatches between a transcription and an audio signal in automatic speech recognition systems, the system comprising:
a processor; and
a memory in communication with the processor, the memory storing instructions that, when executed by the processor, causes the processor to:
create an alignment of the audio signal with the reference transcription;
determine a score using log Deep Neural Network probabilities for every word in each utterance of the audio signal;
sort the words in order of their associated scores; and
remove the utterances containing problematic words that do not meet a threshold.
