# US 2020/0402500 A1 · Method and device for generating speech recognition model and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0402500 A1
- Document ID: US20200402500A1
- Status: Published application
- Kind: A1
- Application number: 17/011,809
- Filed: 2020-09-03
- Published: 2020-12-24
- Earliest claimed: 2019-09-06
- Assignees: Beijing Dajia Internet Information Technology Co., Ltd.
- Applicants: Beijing Dajia Internet Information Technology Co., Ltd.
- Inventors: Yuanyuan Zhao; Jie Li; Xiaorui Wang; Yan Li
- Web page: https://patentgrep.com/patent/US20200402500A1

## CPC

- G10L 15/063: {Training}
- G10L 15/02: Feature extraction for speech recognition; Selection of recognition unit
- G10L 15/16: using artificial neural networks
- G10L 2015/027: {Syllables being the recognition units}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A method and device for generating speech recognition model are provided. The method includes: obtaining training samples, wherein each training sample includes a speech frame sequence and a labeled text sequence; training the encoder by using the speech frame sequence as an input feature and using speech encoded frames of the speech frame sequence as an output feature; training the decoder by using the speech encoded frames as a first input feature and using the labeled text sequence as a first output feature, and obtaining a current prediction text sequence; and training the decoder again by using the speech encoded frames as a second input feature and using a sequence as a second output feature, wherein the sequence is obtained by sampling the labeled text sequence and the current prediction text sequence based on a preset probability.

## Claims

### Claim 1

A method for generating a speech recognition model, wherein the speech recognition model comprises an encoder and a decoder, and the method comprises:
obtaining training samples, wherein each of the training samples comprises a speech frame sequence and a corresponding labeled text sequence;
training the encoder by using the speech frame sequence as an input feature of the encoder and using speech encoded frames of the speech frame sequence as an output feature of the encoder;
training the decoder by using the speech encoded frames as a first input feature of the decoder and using the labeled text sequence as a first output feature of the decoder, and obtaining a current prediction text sequence; and
training the decoder again by using the speech encoded frames as a second input feature of the decoder and using a sequence as a second output feature of the decoder, wherein the sequence is obtained by sampling the labeled text sequence and the current prediction text sequence based on a preset probability.

### Claim 2 (depends on 1)

The method of claim 1, wherein said obtaining training samples comprises:
obtaining a speech signal;
obtaining an initial speech frame sequence by extracting speech features from the speech signal;
obtaining spliced speech frames by splicing speech frames in the initial speech frame sequence; and
obtaining the speech frame sequence by down-sampling the spliced speech frames.

### Claim 3 (depends on 1)

The method of claim 1, wherein the preset probability is determined based on an accuracy of the current prediction text sequence output by the decoder.

### Claim 4 (depends on 3)

The method of claim 3, wherein the preset probability is determined by:
determining the preset probability of sampling the current prediction text sequence in a direct proportion to the accuracy of the current prediction text sequence;
determining the preset probability of sampling the labeled text sequence in an inverse proportion to the accuracy of the current prediction text sequence.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
terminating training the speech recognition model in response to that a proximity between the current prediction text sequence and the labeled text sequence satisfies a preset value and that a character error rate in the current prediction text sequence satisfies a preset value, wherein the labeled text sequence corresponds to the current prediction text sequence.

### Claim 6 (depends on 1)

The method of claim 1, wherein the labeled text sequence is a labeled syllable sequence, and the prediction text sequence is a predicted syllable sequence.

### Claim 7

A device for generating a speech recognition model, wherein the speech recognition model comprises an encoder and a decoder, and the device comprises:
a processor; and
a memory configured to store instructions executable by the processor;
wherein the processor is configured to execute the instructions to:
obtain training samples, wherein each of the training sample comprises a speech frame sequence and a corresponding labeled text sequence;
train the encoder by using the speech frame sequence as an input feature of the encoder and using speech encoded frames of the speech frame sequence as an output feature of the encoder; and
train the decoder by using the speech encoded frames as a first input feature of the decoder and using the labeled text sequence as a first output feature of the decoder, and obtain a current prediction text sequence;
train the decoder again by using the speech encoded frame as a second input feature of the decoder and using a sequence based on a preset probability as a second output feature of the decoder, wherein the sequence is obtained by sampling the labeled text sequence and the current prediction text sequence based on a preset probability.

### Claim 8 (depends on 7)

The method of claim 7, wherein the processor configured to:
obtain a speech signal;
obtain an initial speech frame sequence by extracting speech features from the speech signal;
obtain spliced speech frames by splicing speech frames in the initial speech frame sequence; and
obtain the speech frame sequence by down-sampling the spliced speech frames.

### Claim 9 (depends on 7)

The method of claim 7, wherein the preset probability is determined based on an accuracy of the current prediction text sequence output by the decoder.

### Claim 10 (depends on 9)

The method of claim 9, wherein processor is configured to:
determine the preset probability of sampling the current prediction text sequence in a direct proportion to the accuracy of the current prediction text sequence output by the decoder, and determine the preset probability of sampling the labeled text sequence in an inverse proportion to the accuracy of the current prediction text sequence output by the decoder.

### Claim 11 (depends on 7)

The method of claim 7, wherein the processor is further configured to:
terminate training the speech recognition model in response to that a proximity between the current prediction text sequence and the labeled text sequence satisfies a preset value and that a character error rate (CER) in the current prediction text sequence satisfies a preset value, wherein the labeled text sequence corresponds to the current prediction text sequence.

### Claim 12 (depends on 7)

The method of claim 7, wherein the labeled text sequence is the labeled syllable sequence, and the prediction text sequence is a predicted syllable sequence.

### Claim 13

A computer readable storage medium storing computer programs that, when executed by a processor, cause the processor to perform the operation of:
obtaining training samples, wherein each of the training samples comprises a speech frame sequence and a corresponding labeled text sequence;
training an encoder by using the speech frame sequence as an input feature of the encoder and using speech encoded frames of the speech frame sequence as an output feature of the encoder;
training a decoder by using the speech encoded frames as a first input feature of the decoder and using the labeled text sequence as a first output feature of the decoder, and obtaining a current prediction text sequence; and
training the decoder again by using the speech encoded frames as a second feature of the decoder and using a sequence as a second output feature of the decoder, wherein the sequence is obtained by sampling the labeled text sequence and the current prediction text sequence based on a preset probability.
