# US 2021/0343274 A1 · Fast and Robust Unsupervised Contextual Biasing for Speech Recognition

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0343274 A1
- Document ID: US20210343274A1
- Status: Published application
- Kind: A1
- Application number: 16/993,797
- Filed: 2020-08-14
- Published: 2021-11-04
- Earliest claimed: 2020-05-01
- Assignees: salesforce.com, inc.
- Applicants: salesforce.com, inc.
- Inventors: Young Mo Kang; Yingbo Zhou
- Web page: https://patentgrep.com/patent/US20210343274A1

## CPC

- G10L 15/063: {Training}
- G10L 15/197: Probabilistic grammars, e.g. word n-grams
- G10L 15/16: using artificial neural networks
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G10L 2015/0631: {Creating reference templates; Clustering}
- G10L 2015/088: {Word spotting}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

An automatic speech recognition (ASR) system that determines a textual representation of a word from a word spoken in a natural language is provided. The ASR system uses an acoustic model, a language model, and a decoder. When the ASR system receives a spoken word, the acoustic model generates word candidates for the spoken word. The language model determines an n-gram score for each word candidate. The n-gram score includes a base score and a bias score. The bias score is based on a logarithmic probability of the word candidate, where the logarithmic probability is derived using a class-based language model where the words are clustered into non-overlapping clusters according to word statistics. The decoder decodes a textual representation of the spoken word from the word candidates and the corresponding n-gram score for each word candidate.

## Claims

### Claim 1

An automatic speech recognition (ASR) system that determines a textual representation of a word spoken in a natural language, the system comprising:
a language model configured to:
for each word candidate in word candidates that correspond to a spoken word:
determine a base score;
determine a bias score from a logarithmic probability associated with the each word candidate, wherein the logarithmic probability for the each word candidate is determined from a class-based language model; and
combine the base score and the bias score into an n-gram score; and
a decoder neural network configured to determine the textual representation of the spoken word using n-gram scores associated with the word candidates.

### Claim 2 (depends on 1)

The ASR system of claim 1, wherein the language model is configured to increase the base score with the bias score for the each word candidate that is an out-of-vocabulary word, wherein the out-of-vocabulary word is a word that is not included in a vocabulary used to train the language model.

### Claim 3 (depends on 1)

The ASR system of claim 1, wherein the language model is configured to decrease the base score with the bias score for the each word candidate that is a vocabulary word, wherein the vocabulary word is a word that is included in a vocabulary used to train the language model.

### Claim 4 (depends on 1)

The ASR system of claim 1, further configured to:
build the class-based language model using a second vocabulary of words, wherein the class-based language model has a plurality of non-overlapping classes of words according to n-gram statistics for the words in the second vocabulary of words;
determine a logarithmic probability for each word in the second vocabulary of words from the class-based language model, wherein the logarithmic probability is based on a class in the non-overlapping classes; and
store the logarithmic probability for the each word in a memory accessible to the language model.

### Claim 5 (depends on 1)

The ASR system of claim 1, further comprising:
an acoustic model configured to receive the spoken word and determine the word candidates for the spoken word.

### Claim 6 (depends on 5)

The ASR system of claim 5, wherein the acoustic model is further configured to determine an acoustic score for each word candidate in the word candidates; and
the decoder neural network is further configured to determine the textual representation of the spoken word using acoustic scores associated with the word candidates.

### Claim 7 (depends on 1)

The ASR system of claim 1, wherein the bias score does not depend on previous words received by the ASR system and the base score depends on the previous words received by the ASR system.

### Claim 8

A method for determining a textual representation of a word spoken in a natural language, the method comprising:
for each word candidate in word candidates that correspond to a spoken word:
determining, using a language model stored in memory and executed on a processor, a base score;
determine, using the language model, a bias score from a logarithmic probability associated with the each word candidate, wherein the logarithmic probability for the each word candidate is pre-determined using a class-based language model; and
combining the base score and the bias score into an n-gram score; and
determining, using a decoder neural network, the textual representation of the spoken word using n-gram scores associated with the word candidates.

### Claim 9 (depends on 8)

The method of claim 8, further comprising:
increasing, using the language model, the base score with the bias score for the each word candidate that is an out-of-vocabulary word, wherein the out-of-vocabulary word is a word that is not included in a vocabulary used to train the language model.

### Claim 10 (depends on 8)

The method of claim 8, further comprising:
decreasing the base score with the bias score for the each word candidate that is a vocabulary word, wherein the vocabulary word is a word that is included in a vocabulary used to train the language model.

### Claim 11 (depends on 8)

The method of claim 8, further comprising:
prior to determining the textual representation for the spoken word, building the class-based language model using a second vocabulary of words, wherein the class-based language model has a plurality of non-overlapping classes of words according to n-gram statistics for the words in the second vocabulary of words;
determining a logarithmic probability for each word in the second vocabulary of words using the non-overlapping classes in the class-based language model;
storing the logarithmic probability for the each word in a memory accessible to the language model; and
deleting the class-based language model.

### Claim 12 (depends on 8)

The method of claim 8, further comprising:
receiving, at an acoustic model, the spoken word and determine the word candidates for the spoken word.

### Claim 13 (depends on 8)

The method of claim 8, further comprising:
determining, using an acoustic model, an acoustic score for each word candidate in the word candidates; and
determining, using the neural network decoder, the textual representation of the spoken word using acoustic scores associated with the word candidates.

### Claim 14 (depends on 8)

The method of claim 8, wherein the bias score does not depend on previous words received by an ASR system and the base score depends on the previous words received by the ASR system.

### Claim 14

An automatic speech recognition (ASR) system that determines a textual representation of a word spoken in a natural language, the system comprising:
build a class-based language model using a vocabulary of words, wherein the class-based language model has a plurality of non-overlapping classes of words according to n-gram statistics for the words in the vocabulary of words;
determine a logarithmic probability for each word in the vocabulary of words using the non-overlapping classes in the class-based language model;
a language model configured to determine a base score and a bias score for at least one word candidate for a spoken word, wherein the bias score is based on a logarithmic probability for the at least one word candidate; and
a decoder configured to convert the at least one word candidate into a textual representation of the spoken word based on a combination of the base score and the bias score.

### Claim 15 (depends on 14)

The ASR system of claim 14, wherein the language model is configured to increase the base score with the bias score for the at least one word candidate that is an out-of-vocabulary word, wherein the out-of-vocabulary word is a word that is not included in a vocabulary used to train the language model.

### Claim 16 (depends on 14)

The ASR system of claim 14, wherein the language model is configured to decrease the base score with the bias score for the at least one word candidate that is a vocabulary word, wherein the vocabulary word is a word that is included in a vocabulary used to train the language model.

### Claim 17 (depends on 14)

The ASR system of claim 14, further configured to:
delete the class-based language model after the logarithmic probability for each word in the vocabulary of words has been determined.

### Claim 18 (depends on 14)

The ASR system of claim 14, further comprising:
an acoustic model configured to receive the spoken word and determine the at least one word candidate for the spoken word and an acoustic score for the at least one word candidate.

### Claim 19 (depends on 14)

The ASR system of claim 14, wherein the decoder neural network is further configured to determine the textual representation of the spoken word using acoustic scores associated with the at least one word candidate and the combination of the base score and the bias score.

### Claim 20 (depends on 14)

The ASR system of claim 14, wherein the bias score does not depend on previous words processed by the ASR system and the base score depends on the previous words processed by the ASR system.
