# US 2002/0177999 A1 · Method and apparatus for evaluating the accuracy of a speech recognition system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2002/0177999 A1
- Document ID: US20020177999A1
- Status: Published application
- Kind: A1
- Application number: 09/304,812
- Filed: 1999-05-04
- Published: 2002-11-28
- Earliest claimed: 1999-05-04
- Assignees: International Business Machines Corporation
- Inventors: KERRY A. ORTEGA
- Web page: https://patentgrep.com/patent/US20020177999A1

## CPC

- G10L 15/01: Assessment or evaluation of speech recognition systems
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A method and system for evaluating the accuracy of a computer speech recognition system counts and indexes the total number of words dictated and the number of words corrected. The corrections are tallied after being made in a correction window and include words contained in an alternative list as well as words input by the user and within a stored word database. A processor calculates the approximate accuracy of the speech recognition system as the ratio of the number of correct words to the total number of words dictated. An accuracy ratio is calculated for each dictation session and an overall ratio is calculated for all sessions combined. The system also keeps individual and overall indexes of the number of times the corrected words were in alternate lists or not within the word database and uses these indexes to calculate additional accuracy values.

## Claims

### Claim 1

In a computer system for speech recognition, a method for evaluating how accurately dictated words are recognized during a dictation session by a speech recognition system, comprising the steps of:
counting said dictated words to create a total word index;
counting mis-recognized words to create a correction index;
calculating an accuracy value according to said total word and correction indexes.

### Claim 2 (depends on 1)

The method according to claim 1, wherein said accuracy value is calculated by subtracting said correction index from said total word index to create a recognition index and computing a ratio of said recognition index to said total word index.

### Claim 3 (depends on 2)

The method according to claim 2, wherein said total word and recognition indexes are summed for multiple dictation sessions and said accuracy value is a ratio of said summed recognition index to said summed total word index.

### Claim 4 (depends on 2)

The method according to claim 2, further comprising the steps of:
comparing each said mis-recognized word to at least one alternate word;
counting each time said mis-recognized words are corrected by said at least one alternate words so as to create an alternate index; and
updating said total word index according to said at least one alternate word.

### Claim 5 (depends on 4)

The method according to claim 4, wherein said accuracy value is calculated by adding said alternate index to said recognition index to create an alternate recognition index and computing a ratio of said alternate recognition index to said total word index.

### Claim 6 (depends on 5)

The method according to claim 5, wherein said total word and alternate recognition indexes are summed for multiple dictation sessions and said accuracy value is a ratio of said summed alternate recognition index to said summed total word index.

### Claim 7 (depends on 5)

The method according to claim 5, further comprising the step of receiving corrected words from user input, wherein said total word index is updated according to said corrected words.

### Claim 8 (depends on 7)

The method according to claim 7, further comprising the steps of:
searching a word database for each said user corrected word; and
counting each time said user corrected word is not within said word database so as to create an out-of-vocabulary index.

### Claim 9 (depends on 8)

The method according to claim 8, wherein said accuracy value is calculated by adding said out-of-vocabulary index to said alternate recognition index to create a vocabulary recognition index and computing a ratio of said vocabulary recognition index to said total word index.

### Claim 10 (depends on 9)

The method according to claim 9, wherein said total word and vocabulary indexes are summed for multiple dictation sessions and said accuracy value is a ratio of said summed vocabulary recognition index to said summed total word index.

### Claim 11

A system for evaluating the accuracy in which dictated words are recognized during a dictation session by a computer speech recognition system, comprising:
counting means for determining the number of said dictated words so as to create a total word index and for determining the number of mis-recognized words so as to create a correction index;
calculating means for calculating an accuracy value according to said total word and correction indexes.

### Claim 12 (depends on 11)

The system as recited in claim 11, wherein said calculating means calculates said accuracy value by subtracting said correction index from said total word index to create a recognition index and computing a ratio of said recognition index to said total word index.

### Claim 13 (depends on 12)

The system as recited in claim 12, further comprising a comparing means and an updating means, wherein:
said comparing means compares each said mis-recognized word to at least one alternate word;
said counting means counts each time said mis-recognized words are corrected by said alternate words so as to create an alternate index; and
said updating means updates said total word index according to said alternate words.

### Claim 14 (depends on 13)

The system as recited in claim 13, wherein said calculating means calculates said accuracy value by adding said alternate index to said recognition index to create an alternate recognition index and computing a ratio of said alternate recognition index to said total word index.

### Claim 15 (depends on 13)

The system as recited in claim 13, further comprising an input means for receiving corrected words from a user, wherein said total word index is updated according to said user corrected words.

### Claim 16 (depends on 15)

The system as recited in claim 15, further comprising a searching means, wherein:
said searching means searches a word database for each said user corrected word; and
said counting means counts each time said user corrected word is not within said word database so as to create an out-of-vocabulary index.

### Claim 17 (depends on 16)

The system as recited in claim 16, wherein said calculation means calculates said accuracy value by adding said out-of-vocabulary index to said alternate recognition index to create a vocabulary recognition index and computing a ratio of said vocabulary recognition index to said total word index.

### Claim 18

A machine readable storage, having stored thereon a computer program having a plurality of code sections executable by a machine for causing the machine to perform the steps of:
counting dictated words to create a total word index;
counting mis-recognized words to create a correction index;
calculating an accuracy value according to said total word and correction indexes.
