# US 2024/0203397 A1 · Systems and methods for improved automatic speech recognition systems

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0203397 A1
- Document ID: US20240203397A1
- Status: Published application
- Kind: A1
- Application number: 18/066,174
- Filed: 2022-12-14
- Published: 2024-06-20
- Earliest claimed: 2022-12-14
- Assignees: Comcast Cable Communications, LLC
- Applicants: Comcast Cable Communications, LLC
- Inventors: Raphael TANG; Karun KUMAR; Kendra CHALKLEY; Liming ZHANG; Wenyan LI; Pamela SHAPIRO; Yajie MAO; Gefei YANG; Jun Ho SHIN; Geoffrey Craig MURRAY
- Web page: https://patentgrep.com/patent/US20240203397A1

## CPC

- G10L 15/01: Assessment or evaluation of speech recognition systems
- G06F 40/169: Annotation, e.g. comment data or footnotes
- G10L 15/063: {Training}
- G10L 15/197: Probabilistic grammars, e.g. word n-grams
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Selection of training utterances may be carried out in a sample-efficient manner, and the selected training utterances may be annotated to provide improved training information to an ASR system. A computing device may receive, from an ASR system, one or more transcript-score pairs, wherein a transcript-score pair comprises a transcription associated with a voice query and at least one score associated with the transcription. The computing device may determine a likelihood of a word error associated with each transcription of the one or more transcript-score pairs. The computing device may determine, based on the likelihood of the word error, an effect on a word-error rate of the ASR system. The computing device may send at least one of the one or more transcript-score pairs with a threshold effect on the word-error rate of the ASR system to be annotated.

## Claims

### Claim 1

A method comprising:
receiving, at a computing device and from an automatic speech recognition system, one or more transcript-score pairs, wherein a transcript-score pair comprises a transcription associated with a voice query and at least one score associated with the transcription;
determining a likelihood of a word error associated with each transcription of the one or more transcript-score pairs, wherein determining the likelihood of the word error comprises determining a difference between a transcription and a true annotation associated with the voice query;
determining, based on the likelihood of the word error, an effect on a word-error rate of the automatic speech recognition system associated with each one of the one or more transcript-score pairs; and
sending at least one of the one or more transcript-score pairs with a threshold effect on the word-error rate of the automatic speech recognition system to be annotated.

### Claim 2 (depends on 1)

The method of claim 1, further comprising determining, based on a confidence associated with the automatic speech recognition system, a word-error rate estimator associated with the automatic speech recognition system.

### Claim 3 (depends on 1)

The method of claim 1, wherein the likelihood of the word error comprises a likelihood of the automatic speech recognition system producing an incorrect transcription of the voice query, and wherein the effect on the word-error rate of the automatic speech recognition system comprises a proportion of an overall word-error rate of the automatic speech recognition caused by the likelihood of the word error.

### Claim 4 (depends on 1)

The method of claim 1, further comprising determining a probability of a correct transcription of each one of the one or more transcript-score pairs associated with the voice query.

### Claim 5 (depends on 4)

The method of claim 4, wherein the determining the probability associated with each one of the one or more transcript-score pairs further comprises a scaling factor, wherein the scaling factor is a temperature hyperparameter.

### Claim 6 (depends on 1)

The method of claim 1, wherein each one of the one or more transcript-score pairs comprises a determined transcription associated with the voice query and a confidence score associated with a likelihood the determined transcription matches the voice query.

### Claim 7 (depends on 1)

The method of claim 1, further comprising updating, based on the sending the at least one of the one or more transcript-score pairs to be annotated, the automatic speech recognition system.

### Claim 8 (depends on 7)

The method of claim 7, wherein updating the automatic speech recognition system further comprises training, based on the at least one of the one or more transcript-score pairs, the automatic speech recognition system.

### Claim 9

A method comprising:
receiving, at an automatic speech recognition system, a plurality of voice queries;
determining, by the automatic speech recognition system, at least one transcript-score pair associated with each of the plurality of voice queries, wherein a transcript-score pair comprises a transcription associated with a voice query of the plurality of voice queries and at least one score associated with the transcription;
receiving, at a computing device, each of the at least one transcript-score pairs associated with each of the plurality of voice queries;
determining an effect on a word-error rate of the automatic speech recognition system associated with each one of the at least one transcript-score pairs; and
sending a subset of the at least one transcript-score pairs with a threshold effect on the word-error rate of the automatic speech recognition system to be annotated.

### Claim 10 (depends on 9)

The method of claim 9, wherein the effect on the word-error rate of the automatic speech recognition system comprises a proportion of an overall word-error rate of the automatic speech recognition caused by the subset of the at least one transcript-score pairs.

### Claim 11 (depends on 9)

The method of claim 9, further comprising determining a probability of a correct transcription of each one of the one or more transcript-score pairs associated with the voice query.

### Claim 12 (depends on 11)

The method of claim 11, wherein the determining the probability associated with each one of the one or more transcript-score pairs further comprises a scaling factor, wherein the scaling factor is a temperature hyperparameter.

### Claim 13 (depends on 9)

The method of claim 9, wherein each one of the one or more transcript-score pairs comprise a determined transcription associated with the voice query and a confidence score associated with a likelihood the determined transcription matches the voice query.

### Claim 14 (depends on 9)

The method of claim 9, further comprising updating, based on the sending the subset of the at least one transcript-score pairs to be annotated, the automatic speech recognition system.

### Claim 15 (depends on 14)

The method of claim 14, wherein updating the automatic speech recognition system further comprises training, based on the at least one of the one or more transcript-score pairs, the automatic speech recognition system.

### Claim 16

A system comprising:
an automatic speech recognition system configured to:
receive a voice query; and
send, to a computing device, at least one transcript-score pair associated with the voice query, wherein the at least one transcript-score pair comprises a transcription associated with the voice query and at least one score associated with the transcription; and
the computing device configured to:
receive each of the at least one transcript-score pairs associated with the voice query;
determine an effect on a word-error rate of the automatic speech recognition system associated with each one of the at least one transcript-score pairs; and
send one or more of the at least one transcript-score pairs with a threshold effect on the word-error rate of the automatic speech recognition system to be annotated.

### Claim 17 (depends on 16)

The system of claim 16, wherein the effect on the word-error rate of the automatic speech recognition system comprises a proportion of an overall word-error rate of the automatic speech recognition caused by the one or more of the at least one transcript-score pairs with the threshold effect on the word-error rate.

### Claim 18 (depends on 16)

The system of claim 16, further comprising determining a probability of a correct transcription of each one of the one or more transcript-score pairs associated with the voice query.

### Claim 19 (depends on 18)

The system of claim 18, wherein the determining the probability associated with each one of the one or more transcript-score pairs further comprises a scaling factor, wherein the scaling factor is a temperature hyperparameter.

### Claim 20 (depends on 16)

The system of claim 16, wherein each one of the one or more transcript-score pairs comprise a determined transcription associated with the voice query and a confidence score associated with a likelihood the determined transcription matches the voice query.

### Claim 21 (depends on 16)

The system of claim 16, further comprising training, based on the sending the one or more of the at least one transcript-score pairs to be annotated, the automatic speech recognition system.
