# US 2014/0156275 A1 · Method of Active Learning for Automatic Speech Recognition

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2014/0156275 A1
- Document ID: US20140156275A1
- Status: Published application
- Kind: A1
- Application number: 14/176,439
- Filed: 2014-02-10
- Published: 2014-06-05
- Earliest claimed: 2002-07-29
- Assignees: AT&T Intellectual Property II, L.P.
- Applicants: AT&T Intellectual Property II, L.P.
- Inventors: Allen Louis Gorin; Dilek Z. Hakkani-Tur; Giuseppe Riccardi
- Web page: https://patentgrep.com/patent/US20140156275A1

## CPC

- G10L 15/063: {Training}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

State-of-the-art speech recognition systems are trained using transcribed utterances, preparation of which is labor-intensive and time-consuming. The present invention is an iterative method for reducing the transcription effort for training in automatic speech recognition (ASR). Active learning aims at reducing the number of training examples to be labeled by automatically processing the unlabeled examples and then selecting the most informative ones with respect to a given cost function for a human to label. The method comprises automatically estimating a confidence score for each word of the utterance and exploiting the lattice output of a speech recognizer, which was trained on a small set of transcribed data. An utterance confidence score is computed based on these word confidence scores; then the utterances are selectively sampled to be transcribed using the utterance confidence scores.

## Claims

### Claim 1

A method comprising:
recognizing untranscribed audio utterances that are candidates for transcription using a trained acoustic model and a trained language model;
computing confidence scores associated with an accuracy of speech recognition of the untranscribed audio utterances; and
transcribing an audio utterance from the untranscribed audio utterances, the audio utterance having a lowest confidence score in the confidence scores.

### Claim 2 (depends on 1)

The method of claim 1, further comprising iteratively repeating the transcribing with additional audio utterances from the untranscribed audio utterances until a word accuracy converges.

### Claim 3 (depends on 1)

The method of claim 1, further comprising removing the audio utterance which was transcribed from a database comprising the untranscribed audio utterances.

### Claim 4 (depends on 1)

The method of claim 1, wherein the audio utterance comprises a plurality of audio works.

### Claim 5 (depends on 1)

The method of claim 1, further comprising leaving out from consideration for transcription utterances with confidence scores indicating that the untranscribed audio utterances were correctly recognized.

### Claim 6 (depends on 1)

The method of claim 1, wherein word posterior probability estimates are used for the confidence scores associated with the untranscribed audio utterances.

### Claim 7 (depends on 6)

The method of claim 6, wherein a word is considered to be correctly recognized when an associated posterior probability is higher than a second threshold value.

### Claim 8

A system comprising:
a processor; and
a computer-readable storage medium having instructions stored which, when executed by the processor, cause the processor to perform operations comprising:
recognizing untranscribed audio utterances that are candidates for transcription using a trained acoustic model and a trained language model;
computing confidence scores associated with an accuracy of speech recognition of the untranscribed audio utterances; and
transcribing an audio utterance from the untranscribed audio utterances, the audio utterance having a lowest confidence score in the confidence scores.

### Claim 9 (depends on 8)

The system of claim 8, the computer-readable storage medium having additional instructions stored which result in operations comprising iteratively repeating the transcribing with additional audio utterances from the untranscribed audio utterances until a word accuracy converges.

### Claim 10 (depends on 8)

The system of claim 8, the computer-readable storage medium having additional instructions stored which result in operations comprising removing the audio utterance which was transcribed from a database comprising the untranscribed audio utterances.

### Claim 11 (depends on 8)

The system of claim 8, wherein the audio utterance comprises a plurality of audio works.

### Claim 12 (depends on 8)

The system of claim 8, the computer-readable storage medium having additional instructions stored which result in operations comprising leaving out from consideration for transcription utterances with confidence scores indicating that the untranscribed audio utterances were correctly recognized.

### Claim 13 (depends on 8)

The system of claim 8, wherein word posterior probability estimates are used for the confidence scores associated with the untranscribed audio utterances.

### Claim 14 (depends on 13)

The system of claim 13, wherein a word is considered to be correctly recognized when an associated posterior probability is higher than a second threshold value.

### Claim 15

A computer-readable storage device having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising:
recognizing untranscribed audio utterances that are candidates for transcription using a trained acoustic model and a trained language model;
computing confidence scores associated with an accuracy of speech recognition of the untranscribed audio utterances; and
transcribing an audio utterance from the untranscribed audio utterances, the audio utterance having a lowest confidence score in the confidence scores.

### Claim 16 (depends on 15)

The computer-readable storage device of claim 15, having additional instructions stored which result in operations comprising iteratively repeating the transcribing with additional audio utterances from the untranscribed audio utterances until a word accuracy converges.

### Claim 17 (depends on 15)

The computer-readable storage device of claim 15, having additional instructions stored which result in operations comprising removing the audio utterance which was transcribed from a database comprising the untranscribed audio utterances.

### Claim 18 (depends on 15)

The computer-readable storage device of claim 15, wherein the audio utterance comprises a plurality of audio works.

### Claim 19 (depends on 15)

The computer-readable storage device of claim 15, having additional instructions stored which result in operations comprising leaving out from consideration for transcription utterances with confidence scores indicating that the untranscribed audio utterances were correctly recognized.

### Claim 20 (depends on 19)

The computer-readable storage device of claim 19, wherein word posterior probability estimates are used for the confidence scores associated with the untranscribed audio utterances.
