# US 2016/0260428 A1 · Statistical acoustic model adaptation method, acoustic model learning method suitable for statistical acoustic model adaptation, storage medium storing parameters for building deep neural network, and computer program for adapting statistical acoustic model

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0260428 A1
- Document ID: US20160260428A1
- Status: Published application
- Kind: A1
- Application number: 15/031,449
- Filed: 2014-11-06
- Published: 2016-09-08
- Earliest claimed: 2013-11-27
- Assignees: National Institute of Information and Communications Technology
- Applicants: National Institute of Information and Communications Technology
- Inventors: Shigeki MATSUDA; Xugang LU
- Web page: https://patentgrep.com/patent/US20160260428A1

## CPC

- G10L 15/16: using artificial neural networks
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/045: Combinations of networks
- G06N 3/0499: Feedforward networks
- G06N 3/08: Learning methods
- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06N 3/09: Supervised learning
- G06N 3/096: Transfer learning
- G10L 15/075: {supervised, i.e. under machine guidance}
- G10L 15/063: {Training}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

[Object] An object is to provide a statistical acoustic model adaptation method capable of efficient adaptation of an acoustic model using DNN with training data under a specific condition and achieving higher accuracy. [Solution] A method of speaker adaptation of an acoustic model using DNN includes the steps of: storing speech data 90 to 98 of different speakers separately in a first storage device; preparing speaker-by-speaker hidden layer modules 112 to 120; performing preliminary learning of all layers 42, 44, 110, 48, 50, 52 and 54 of a DNN 80 by switching and selecting the speech data 90 to 98 while dynamically replacing a specific layer 110 with hidden layer modules 112 to 120 corresponding to the selected speech data; replacing the specific layer 110 of the DNN that has completed the preliminary learning with an initial hidden layer; and training the DNN with speech data of a specific speaker while fixing parameters of layers other than the initial hidden layer.

## Claims

### Claim 1

A statistical acoustic model adaptation method for speech recognition under a specific condition, wherein
said acoustic model is an acoustic model using Deep Neural Network or DNN, said DNN including a plurality of layers of three or more;
said method comprising the steps of:
a computer readable first storage device separately storing speech data under a plurality of conditions;
a computer preparing a plurality of hidden layer modules for respective conditions corresponding to said plurality of conditions;
the computer, while switching and selecting speech data under said plurality of conditions, performing preliminary learning of all layers of said DNN while dynamically replacing a specific one of said plurality of layers with a hidden layer module corresponding to the selected speech data;
the computer replacing said specific layer of said DNN that has completed the learning at said step of performing preliminary learning with an initial hidden layer prepared in advance;
a second computer readable storage device storing speech data under a condition as an object of adaptation; and
reading the speech data under the condition of said object of adaptation from said second storage device while fixing parameters of layers other than said initial hidden layer of said DNN produced at said step of replacing, and training said DNN.

### Claim 2 (depends on 1)

The method according to claim 1, further comprising the steps of:
prior to said step of learning, the computer executing condition-independent training of said DNN with a huge amount of speech data not discriminated by any utterance condition; and
storing, in a third storage device, parameters of the hidden layer of said specific layer of said DNN after training at said step of conducting condition-independent training; wherein
said step of replacing includes the step of the computer copying, to parameters of said specific layer of said DNN that has completed learning at said step of preliminary learning, the parameters of the hidden layer stored in said third storage device.

### Claim 3

An acoustic model learning method suitable for statistical acoustic model adaptation under a specific condition, wherein
said acoustic model is an acoustic model using Deep Neural Network or DNN, said DNN including a plurality of layers of three or more;
said method comprising the steps of:
a computer readable first storage device separately storing speech data under a plurality of conditions;
a computer preparing a plurality of hidden layer modules for respective conditions corresponding to said plurality of conditions;
the computer, while switching and selecting speech data under said plurality of conditions, performing preliminary learning of all layers of said DNN while dynamically replacing a specific one of said plurality of layers with a hidden layer module corresponding to the selected speech data; and
the computer storing, in a storage medium, parameters of layers other than said specific layer of said DNN that has completed learning at said step of preliminary learning.

### Claim 4 (depends on 3)

The method according to claim 3, further comprising the steps of:
the computer executing condition-independent training of said DNN with a huge amount of speech data not discriminated by any utterance condition; and
storing parameters of the hidden layer of said specific layer of said DNN after training at said step of conducting condition-independent training additionally to said storage medium.

### Claim 5

A storage medium storing parameters for building a Deep Neural Network or DNN for an acoustic model learning, suitable for statistical acoustic model adaptation for speech recognition under a specific condition, wherein
the parameters stored in said storage medium are for building a DNN including a plurality of layers of three or more;
said storage medium further storing
a program for training, by fixing parameters of layers other than a specific layer of the DNN built by using the parameters stored in said storage medium, said DNN with speech data, whereby training only said specific layer of said DNN; wherein
said specific layer is predetermined corresponding to the DNN stored in said storage medium.

### Claim 6

A computer program for adapting a statistical acoustic model for speech recognition under a specific condition, wherein
said acoustic model is an acoustic model using Deep Neural Network or DNN, said DNN including a plurality of layers of three or more;
said computer program causing a computer to execute the steps of:
separately storing, from a computer readable first storage device, speech data under a plurality of conditions;
preparing a plurality of hidden layer modules for respective conditions corresponding to said plurality of conditions;
while switching and selecting speech data under said plurality of conditions, performing preliminary learning of all layers of said DNN while dynamically replacing a specific one of said plurality of layers with a hidden layer module corresponding to the selected speech data;
replacing said specific layer of said DNN that has completed the learning at said step of performing preliminary learning with an initial hidden layer prepared in advance; and
storing the parameters of DNN obtained at said step of replacing in a storage medium.
