# US 2018/0322865 A1 · Artificial intelligence-based acoustic model training method and apparatus, device and storage medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2018/0322865 A1 - Document ID: US20180322865A1 - Status: Published application - Kind: A1 - Application number: 15/961,724 - Filed: 2018-04-24 - Published: 2018-11-08 - Earliest claimed: 2017-05-05 - Assignees: Baidu Online Network Technology (Beijing) Co., Ltd. - Applicants: Baidu Online Network Technology(Beijing) Co Ltd - Inventors: Bin HUANG; Yiping PENG - Web page: https://patentgrep.com/patent/US20180322865A1 ## CPC - G10L 15/063: {Training} - G10L 15/16: using artificial neural networks - G10L 15/02: Feature extraction for speech recognition; Selection of recognition unit - G10L 2015/0631: {Creating reference templates; Clustering} - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract The present disclosure provides an artificial intelligence-based acoustic model training method and apparatus, a device and a storage medium, wherein the method comprises: obtaining manually-annotated speech data; training according to the manually-annotated speech data to obtain a first acoustic model; obtaining unannotated speech data; training according to the unannotated speech data and the first acoustic model to obtain a desired second acoustic model. The solution of the present disclosure can be applied to save manpower costs and improve the training efficiency. ## Claims ### Claim 1 An artificial intelligence-based acoustic model training method, wherein the method comprises: obtaining manually-annotated speech data; training according to the manually-annotated speech data to obtain a first acoustic model; obtaining unannotated speech data; training according to the unannotated speech data and the first acoustic model to obtain a desired second acoustic model. ### Claim 2 (depends on 1) The method according to claim 1, wherein a type of each of the first acoustic model and the second acoustic model comprises an acoustic model having a hybrid structure of a convolutional neural network and a recurrent neural network. ### Claim 3 (depends on 1) The method according to claim 1, wherein the training according to the manually-annotated speech data to obtain a first acoustic model comprises: determining an alignment relationship of each speech frame and a manually-annotated syllable state according to the manually-annotated speech data; considering the alignment relationship as a training target and training the first acoustic model based on a first rule to obtain a first acoustic model in an initial state; considering the alignment relationship as a training target and further training the first acoustic model in the initial state based on a second rule to obtain the first acoustic model. ### Claim 4 (depends on 3) The method according to claim 3, wherein the training according to the unannotated speech data and the first acoustic model to obtain a desired second acoustic model comprises: inputting the unannotated speech data to the first acoustic model to obtain the alignment relationship of each speech frame and a corresponding syllable state output by the first acoustic model; considering the alignment relationship as a training target and training the second acoustic model based on a first rule to obtain a second acoustic model in an initial state; considering the alignment relationship as a training target and further training the second acoustic model in the initial state based on a second rule to obtain the second acoustic model. ### Claim 5 (depends on 4) The method according to claim 4, wherein the first rule comprises a Cross Entropy CE rule; the second rule comprises a Connectionist Temporal Classification CTC rule. ### Claim 6 (depends on 1) The method according to claim 1, wherein before training according to the manually-annotated speech data to obtain a first acoustic model, the method further comprises: extracting an acoustic feature from the manually-annotated speech data; the training according to the manually-annotated speech data to obtain a first acoustic model comprises: training according to the acoustic feature extracted from the manually-annotated speech data to obtain the first acoustic model; before training according to the unannotated speech data and the first acoustic model to obtain a desired second acoustic model, the method further comprises: extracting the acoustic feature from the unannotated speech data; the training according to the unannotated speech data and the first acoustic model to obtain a desired second acoustic model comprises: training according to the acoustic feature extracted from the unannotated speech data and the first acoustic model to obtain the second acoustic model. ### Claim 7 (depends on 6) The method according to claim 6, wherein the extracting an acoustic feature from a speech data comprises: performing Fast Fourier Transformation FFT for the speech data with a preset first time length as a frame length and with a preset second time length as a frame shift; extracting an Mel-scale Filter Bank thank acoustic feature according to an FFT transformation result. ### Claim 8 A computer device, comprising a memory, a processor and a computer program which is stored on the memory and runs on the processor, wherein the processor, upon executing the program, implements the following operation: obtaining manually-annotated speech data; training according to the manually-annotated speech data to obtain a first acoustic model; obtaining unannotated speech data; training according to the unannotated speech data and the first acoustic model to obtain a desired second acoustic model. ### Claim 9 (depends on 8) The computer device according to claim 8, wherein a type of each of the first acoustic model and the second acoustic model comprises an acoustic model having a hybrid structure of a convolutional neural network and a recurrent neural network. ### Claim 10 (depends on 8) The computer device according to claim 8, wherein the training according to the manually-annotated speech data to obtain a first acoustic model comprises: determining an alignment relationship of each speech frame and a manually-annotated syllable state according to the manually-annotated speech data; considering the alignment relationship as a training target and training the first acoustic model based on a first rule to obtain a first acoustic model in an initial state; considering the alignment relationship as a training target and further training the first acoustic model in the initial state based on a second rule to obtain the first acoustic model. ### Claim 11 (depends on 10) The computer device according to claim 10, wherein the training according to the unannotated speech data and the first acoustic model to obtain a desired second acoustic model comprises: inputting the unannotated speech data to the first acoustic model to obtain the alignment relationship of each speech frame and a corresponding syllable state output by the first acoustic model; considering the alignment relationship as a training target and training the second acoustic model based on a first rule to obtain a second acoustic model in an initial state; considering the alignment relationship as a training target and further training the second acoustic model in the initial state based on a second rule to obtain the second acoustic model. ### Claim 12 (depends on 11) The computer device according to claim 11, wherein the first rule comprises a Cross Entropy CE rule; the second rule comprises a Connectionist Temporal Classification CTC rule. ### Claim 13 (depends on 8) The computer device according to claim 8, wherein before training according to the manually-annotated speech data to obtain a first acoustic model, the operation further comprises: extracting an acoustic feature from the manually-annotated speech data; the training according to the manually-annotated speech data to obtain a first acoustic model comprises: training according to the acoustic feature extracted from the manually-annotated speech data to obtain the first acoustic model; before training according to the unannotated speech data and the first acoustic model to obtain a desired second acoustic model, the operation further comprises: extracting the acoustic feature from the unannotated speech data; the training according to the unannotated speech data and the first acoustic model to obtain a desired second acoustic model comprises: training according to the acoustic feature extracted from the unannotated speech data and the first acoustic model to obtain the second acoustic model. ### Claim 14 (depends on 13) The computer device according to claim 13, wherein the extracting an acoustic feature from a speech data comprises: performing Fast Fourier Transformation FFT for the speech data with a preset first time length as a frame length and with a preset second time length as a frame shift; extracting an Mel-scale Filter Bank thank acoustic feature according to an FFT transformation result. ### Claim 15 A computer-readable storage medium on which a computer program is stored, wherein the program, when executed by a processor, implements the following operation: obtaining manually-annotated speech data; training according to the manually-annotated speech data to obtain a first acoustic model; obtaining unannotated speech data; training according to the unannotated speech data and the first acoustic model to obtain a desired second acoustic model. ### Claim 16 (depends on 15) The computer-readable storage medium according to claim 15, wherein a type of each of the first acoustic model and the second acoustic model comprises an acoustic model having a hybrid structure of a convolutional neural network and a recurrent neural network. ### Claim 17 (depends on 15) The computer-readable storage medium according to claim 15, wherein the training according to the manually-annotated speech data to obtain a first acoustic model comprises: determining an alignment relationship of each speech frame and a manually-annotated syllable state according to the manually-annotated speech data; considering the alignment relationship as a training target and training the first acoustic model based on a first rule to obtain a first acoustic model in an initial state; considering the alignment relationship as a training target and further training the first acoustic model in the initial state based on a second rule to obtain the first acoustic model. ### Claim 18 (depends on 17) The computer-readable storage medium according to claim 17, wherein the training according to the unannotated speech data and the first acoustic model to obtain a desired second acoustic model comprises: inputting the unannotated speech data to the first acoustic model to obtain the alignment relationship of each speech frame and a corresponding syllable state output by the first acoustic model; considering the alignment relationship as a training target and training the second acoustic model based on a first rule to obtain a second acoustic model in an initial state; considering the alignment relationship as a training target and further training the second acoustic model in the initial state based on a second rule to obtain the second acoustic model. ### Claim 19 (depends on 18) The computer-readable storage medium according to claim 18, wherein the first rule comprises a Cross Entropy CE rule; the second rule comprises a Connectionist Temporal Classification CTC rule. ### Claim 20 (depends on 15) The computer-readable storage medium according to claim 15, wherein before training according to the manually-annotated speech data to obtain a first acoustic model, the operation further comprises: extracting an acoustic feature from the manually-annotated speech data; the training according to the manually-annotated speech data to obtain a first acoustic model comprises: training according to the acoustic feature extracted from the manually-annotated speech data to obtain the first acoustic model; before training according to the unannotated speech data and the first acoustic model to obtain a desired second acoustic model, the operation further comprises: extracting the acoustic feature from the unannotated speech data; the training according to the unannotated speech data and the first acoustic model to obtain a desired second acoustic model comprises: training according to the acoustic feature extracted from the unannotated speech data and the first acoustic model to obtain the second acoustic model. ### Claim 21 (depends on 20) The computer-readable storage medium according to claim 20, wherein the extracting an acoustic feature from a speech data comprises: performing Fast Fourier Transformation FFT for the speech data with a preset first time length as a frame length and with a preset second time length as a frame shift; extracting an Mel-scale Filter Bank thank acoustic feature according to an FFT transformation result.