# US 8,280,733 B2 · Automatic speech recognition learning using categorization and selective incorporation of user-initiated corrections

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 8,280,733 B2
- Document ID: US8280733B2
- Status: Granted
- Kind: B2
- Application number: 12/884,434
- Filed: 2010-09-17
- Published: 2011-01-20
- Granted: 2012-10-02
- Earliest claimed: 2004-01-20
- Assignees: Microsoft Corporation
- Inventors: Dong Yu; Peter Mau; Mei-Yuh Hwang; Alejandro Acero
- Web page: https://patentgrep.com/patent/US8280733B2

## CPC

- G10L 15/065: Adaptation
- G10L 15/063: {Training}
- G10L 2015/0631: {Creating reference templates; Clustering}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

An automatic speech recognition system recognizes user changes to dictated text and infers whether such changes result from the user changing his/her mind, or whether such changes are a result of a recognition error. If a recognition error is detected, the system uses the type of user correction to modify itself to reduce the chance that such recognition error will occur again. Accordingly, the system and methods provide for significant speech recognition learning with little or no additional user interaction.

## Claims

### Claim 1

A computer-implemented speech recognition system, comprising:
a microphone that receives user speech;
a speech recognition engine coupled to the microphone, the speech recognition engine recognizing the user speech and providing a corresponding textual output on a user interface;
a change recognition component that automatically assigns a categorization to a user-initiated change to the corresponding textual output, the categorization being automatically assigned based at least in part upon a measurement of time indicative of how long it took the user to initiate the change, upon whether or not the user utilized an alternate list to generate the user initiated change, upon an acoustic similarity between the original textual output and the change to the textual output, and upon a number of words that is changed between the original textual output and the change to the textual output; and
wherein automatically assigning the categorization comprises automatically identifying the user-initiated change as being either a correction or an edit operation, wherein the user-initiated change is identified as the correction upon the measurement of time indicating that there was a relatively short amount of time between providing the original textual output and the user initiating the change, wherein the user-initiated change is identified as the correction upon the user utilizing the alternate list, wherein the user-initiated change is identified as the correction upon the original textual output and the change to the textual output being acoustically similar, and wherein the user-initiated change is identified as the correction upon the number of words that is changed between the original textual output and the change to the textual output is determined to be insignificant, and wherein otherwise the categorization is determined to be the edit operation; and
an adaptation component that selectively adapts the speech recognition engine depending upon the categorization.

### Claim 2 (depends on 1)

The system of claim 1, wherein selectively adapting comprises adapting or not adapting depending upon the categorization.

### Claim 3 (depends on 1)

The system of claim 1, wherein selectively adapting comprises adapting or not adapting depending upon whether the categorization is indicates that the user-initiated change is a correction or an edit operation.

### Claim 4 (depends on 1)

The system of claim 1, wherein selectively adapting comprises not adapting if the categorization indicates that the user-initiated correction is not due to a recognition error made by the speech recognition engine.

### Claim 5 (depends on 1)

The system of claim 1, wherein selectively adapting comprises adapting only if the categorization indicates that the user-initiated correction is due to a recognition error made by the speech recognition engine, and wherein adapting comprises adapting so as to make it less likely that the speech recognition engine will subsequently repeat the recognition error during a subsequent recognition operation.

### Claim 6 (depends on 1)

The system of claim 1, wherein automatically assigning the categorization further comprises automatically assigning based at least in part on information received in response to prompting the user for a confirmation.

### Claim 7

A computer-implemented method for adapting a speech recognition system, comprising:
detecting a user-initiated change to dictated text;
automatically assigning a categorization to the user-initiated change to the dictated text, the categorization being automatically assigned based at least in part upon a measurement of time indicative of how long it took the user to initiate the change, wherein automatically assigning the categorization comprises automatically identifying, based at least in part upon the measurement of time, the user-initiated change as being either a correction or an edit operation;
calculating a distance between a pronunciation of the dictated text and a pronunciation of the change to the dictated text;
comparing the calculated distance to a threshold value; and
selectively adapting the speech recognition engine depending upon the categorization and the calculated distance, and wherein selectively adapting further comprises determining a number of times that the pronunciation of the change to the dictated text has occurred and then selectively adapting the speech recognition engine when the number of times exceeds a selected number of times.

### Claim 8 (depends on 7)

The method of claim 7, wherein selectively adapting comprises adapting or not adapting depending upon the categorization.

### Claim 9 (depends on 7)

The method of claim 7, wherein selectively adapting comprises adapting or not adapting depending upon whether the categorization indicates that the user-initiated change is a correction or an edit operation.

### Claim 10 (depends on 7)

The method of claim 7, wherein selectively adapting comprises not adapting if the categorization indicates that the user-initiated correction is not due to a recognition error made by the speech recognition engine.

### Claim 11 (depends on 7)

The method of claim 7, wherein selectively adapting comprises adapting only if the categorization indicates that the user-initiated correction is due to a recognition error made by the speech recognition engine, and wherein adapting comprises adapting so as to make it less likely that the speech recognition engine will subsequently repeat the recognition error during a subsequent recognition operation.

### Claim 12

A computer-implemented method for adapting a speech recognition system, comprising:
detecting a user-initiated change to dictated text;
automatically assigning a categorization to the user-initiated change to the dictated text, the categorization being automatically assigned based at least in part upon a measurement of a number of words changed as a result of the user-initiated change, wherein automatically assigning the categorization comprises automatically identifying, based at least in part upon the measurement of the number of words, the user-initiated change as being either a correction or an edit operation;
determining a number of times and a most recent time that the user-initiated change has occurred; and
selectively adapting the speech recognition engine by temporarily adding a word pair associated with the user-initiated change to a lexicon of the speech recognition engine based at least in part upon the categorization, the number of times the user-initiated change has occurred, and the most recent time that the user-initiated change has occurred, and wherein selectively adapting the speech recognition engine further comprises increasing a probability of a pronunciation associated with the word pair.

### Claim 13 (depends on 12)

The method of claim 12, wherein selectively adapting comprises adapting or not adapting depending upon the categorization.

### Claim 14 (depends on 12)

The method of claim 12, wherein selectively adapting comprises adapting or not adapting depending upon whether the categorization is indicates that the user-initiated change is a correction or an edit operation.

### Claim 15 (depends on 12)

The method of claim 12, wherein selectively adapting comprises not adapting if the categorization indicates that the user-initiated correction is not due to a recognition error made by the speech recognition engine.

### Claim 16 (depends on 12)

The method of claim 12, wherein selectively adapting comprises adapting only if the categorization indicates that the user-initiated correction is due to a recognition error made by the speech recognition engine, and wherein adapting comprises adapting so as to make it less likely that the speech recognition engine will subsequently repeat the recognition error during a subsequent recognition operation.

### Claim 17 (depends on 12)

The method of claim 12, wherein automatically assigning comprises automatically assigning based at least in part on a comparison of a speech recognition score of the dictated text to a speech recognition score of the dictated text after application of the user-initiated change.
