Model learning device, method and recording medium for learning neural network model

A model learning device, comprising:

• an initial value setting part that uses a parameter of a learned first acoustic model including a neural network as a parameter of a second acoustic model including a neural network having a same network structure as the first acoustic model by copying the parameter of the learned first acoustic model;
• a first output probability distribution calculating part that calculates a first output probability distribution including a distribution of an output probability of each unit on an output layer, using features obtained from a learning acoustic signal and the first acoustic model;
• a second output probability distribution calculating part that calculates a second output probability distribution including a distribution of an output probability of each unit on the output layer, using features obtained from a learning acoustic signal and the second acoustic model;
• and
• a modified model update part that calculates a second loss function from a correct unit number corresponding to the learning acoustic signal and from the second output probability distribution, calculates a cross entropy between the first output probability distribution and the second output probability distribution, obtains a weighted sum of the second loss function and the cross entropy, and updates the parameter of the second acoustic model so as to reduce the weighted sum.

A model learning device, comprising:

• an initial value setting part that uses a parameter of a learned first language model including a neural network as a parameter of a second language model including a neural network having a same network structure as the first language model by copying the parameter of the learned first language model;
• a first output probability distribution calculating part that calculates a first output probability distribution including a distribution of an output probability of each unit on an output layer, using a word history that is a word string obtained from learning text data, and the first language model;
• a second output probability distribution calculating part that calculates a second output probability distribution including a distribution of an output probability of each unit on the output layer, using a word history that is a word string obtained from learning text data, and the second language model;
• and
• a modified model update part that calculates a second loss function from a correct word corresponding to the learning word history and from the second output probability distribution, calculates a cross entropy between the first output probability distribution and the second output probability distribution, obtains a weighted sum of the second loss function and the cross entropy, and updates the parameter of the second language model so as to reduce the weighted sum.

A model learning method, comprising:

• an initial value setting step of using a parameter of a learned first model including a neural network as a parameter of a second model including a neural network having a same network structure as the first model by copying the parameter of the learned first model;
• a first output probability distribution calculating step of calculating a first output probability distribution including a distribution of an output probability of each unit on an output layer, using features obtained from learning data and the first model;
• a second output probability distribution calculating step of calculating a second output probability distribution including a distribution of an output probability of each unit on the output layer, using features obtained from the learning data and the second model;
• and
• a modified model update step of calculating a second loss function from correct information corresponding to the learning data and from the second output probability distribution, of calculating a cross entropy between the first output probability distribution and the second output probability distribution, of obtaining a weighted sum of the second loss function and the cross entropy, and of updating the parameter of the second model so as to reduce the weighted sum.

A model learning device, comprising

• an initial value setting part that uses a parameter of a learned first model including a neural network as a parameter of a second model including a neural network having a same network structure as the first model by copying the parameter of the learned first model;
• a first output probability distribution calculating part that calculates a first output probability distribution including a distribution of an output probability of each unit on an output layer, using features obtained from learning data and the first model;
• a second output probability distribution calculating part that calculates a second output probability distribution including a distribution of an output probability of each unit on the output layer, using features obtained from the learning data and the second model;
• and
• a modified model update part that updates the parameter of the second model based on the second output probability distribution, the first output probability distribution, and correct information corresponding to the learning data.

A model learning method, comprising:

• an initial value setting step of using a parameter of a learned first model including a neural network as a parameter of a second model including a neural network having a same network structure as the first model by copying the parameter of the learned first model;
• a first output probability distribution calculating step of calculating a first output probability distribution including a distribution of an output probability of each unit on an output layer, using features obtained from learning data and the first model;
• a second output probability distribution calculating step of calculating a second output probability distribution including a distribution of an output probability of each unit on the output layer, using features obtained from the learning data and the second model;
• and
• a modified model update step of updating the parameter of the second model based on the second output probability distribution, the first output probability distribution, and correct information corresponding to the learning data.