# US 12,183,323 B2 · Method of recognizing speech offline, electronic device, and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,183,323 B2
- Document ID: US12183323B2
- Status: Granted
- Kind: B2
- Application number: 17/644,749
- Filed: 2021-12-16
- Published: 2022-04-07
- Granted: 2024-12-31
- Earliest claimed: 2020-12-23
- Assignees: Beijing Baidu Netcom Science Technology Co., Ltd.
- Applicants: Beijing Baidu Netcom Science Technology Co., Ltd.
- Inventors: Xiaoyin Fu; Mingxin Liang; Zhijie Chen; Qiguang Zang; Zhengxiang Jiang; Liao Zhang; Qi Zhang; Lei Jia
- Web page: https://patentgrep.com/patent/US12183323B2

## CPC

- G10L 15/02: Feature extraction for speech recognition; Selection of recognition unit
- G10L 15/16: using artificial neural networks
- G10L 15/183: using context dependencies, e.g. language models
- G10L 19/032: Quantisation or dequantisation of spectral components
- G10L 2015/027: {Syllables being the recognition units}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

The present disclosure provides a method of recognizing speech offline, electronic device, and a storage medium, relating to a field of artificial intelligence such as speech recognition, natural language processing, and deep learning. The method may include: decoding speech data to be recognized into a syllable recognition result; transforming the syllable recognition result into a corresponding text as a speech recognition result of the speech data.

## Claims

### Claim 1

A method of recognizing speech offline, comprising:
generating a connectionist temporal classification (CTC) output and a listen, attend and spell (LAS) output by using an acoustic model based on feature data extracted from speech data;
decoding the speech data by using a syllable decoder based on the CTC output and the LAS output in combination with a language model, to obtain a syllable recognition result; and
transforming the syllable recognition result into the corresponding text as a speech recognition result of the speech data by using a streaming transformer model;
wherein the acoustic model comprises an encoder, a CTC module and a decoder, the encoder comprises a convolution layer, a first long short term memory (LSTM) layer and a batch normalize (BN) layer, the CTC module comprises a linear layer and a first softmax layer, the decoder is an LAS decoder, and the decoder comprises an attention layer, a second LSTM layer, a layer normalize (LN) layer and a second softmax layer; and
wherein the language model comprises a third LSTM and a fourth LSTM layer.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
compressing a volume of the LSTM language model through 4-bit quantization.

### Claim 3 (depends on 1)

The method of claim 1, wherein the using a streaming transformer model to transform the syllable recognition result into the corresponding text comprises:
adding a time mask operation to each attention structure in the transformer model, and implementing a streaming syllable transformation through the time mask operation.

### Claim 4 (depends on 1)

An electronic device, comprising:
at least one processor; and
a memory communicatively connected to the at least one processor;
wherein the memory stores instructions executable by the at least one processor, and the instructions, when executed by the at least one processor, cause the at least one processor to perform the method of claim 1.

### Claim 5 (depends on 4)

The electronic device of claim 4, wherein the at least one processor is further configured to:
compress a volume of the LSTM language model through 4-bit quantization.

### Claim 6 (depends on 4)

The electronic device of claim 4, wherein the at least one processor is further configured to:
add a time mask operation to each attention structure in the transformer model, and implementing a streaming syllable transformation through the time mask operation.

### Claim 7 (depends on 1)

A non-transitory computer readable storage medium storing computer instructions, wherein the computer instructions are configured to cause a computer to perform the method of claim 1.

### Claim 8 (depends on 7)

The non-transitory computer readable storage medium of claim 7, wherein the computer instructions are configured to cause a computer to:
compress a volume of the LSTM language model through 4-bit quantization.