Method and apparatus for implementing speaker identification neural network
AssigneesSamsung Electronics Co., Ltd., Seoul National University R&Db Foundation
InventorsSungchan Kang, Namsoo Kim, Cheheung Kim, Seokwan Chae
Life of this patent
Estimating the term…
Abstract
A method and apparatus for generating a speaker identification neural network include generating a first neural network that is trained to identify a first speaker with respect to a first voice signal in a first environment, generating a second neural network for identifying a second speaker with respect to a second voice signal in a second environment, and generating the speaker identification neural network by training the second neural network based on a teacher-student training model in which the first neural network is set to a teacher neural network and the second neural network is set to a student neural network.
Claims17
A method of generating a speaker identification neural network, the method comprising:
- generating a first neural network that is trained to identify a first speaker with respect to a first voice signal in a first environment in which a first signal-to-noise (SNR) value of the first voice signal is greater than or equal to a threshold value, wherein the first neural network is configured to set sentence frames by distinguishing a voice part of the first speaker and a silence part in which only noise exists without voice in the first voice signal, and receive the sentence frames of the first voice signal as an input of the first neural network, and identify the first speaker based on a first sentence embedding vector representing a weighted sum of first embedding vectors that are output from a last hidden layer of the first neural network that is provided immediately before an output layer of the first neural network;
- generating a second neural network for identifying a second speaker with respect to a second voice signal in a second environment in which a second SNR value of the second voice signal is less than the threshold value, wherein the second neural network is configured to set sentence frames by distinguishing a voice part of the second speaker and a silence part in which only noise exists without voice in the second voice signal, and receive the sentence frames of the second voice signal as an input of the second neural network, and identify the second speaker based on a second sentence embedding vector indicating a weighted sum of second embedding vectors that are output from a last hidden layer of the second neural network that is provided immediately before an output layer of the second neural network;
- and
- generating the speaker identification neural network by training the second neural network based on a teacher-student training model in which the first neural network is set to a teacher neural network and the second neural network is set to a student neural network,
- wherein the speaker identification neural network comprises an attention layer to adjust the initial weights of the second neural network such that a relatively high weight is assigned to an embedding vector of a period in which a voice signal exists, and a relative low weight is assigned to an embedding vector of a period in which a noise signal exists and any voice signal does not exist, among the second embedding vectors.
An apparatus for generating a speaker identification neural network, the apparatus comprising:
- a memory storing at least one program;
- and
- a processor configured to execute the at least one program to:
- generate a first neural network that is trained to identify a first speaker with respect to a first voice signal in a first environment in which a first signal-to-noise (SNR) value of the first voice signal is greater than or equal to a threshold value, wherein the first neural network is configured to set sentence frames by distinguishing a voice part of the first speaker and a silence part in which only noise exists without voice in the first voice signal, and receive the sentence frames of the first voice signal as an input of the first neural network, and identify the first speaker based on a first sentence embedding vector representing a weighted sum of first embedding vectors that are output from a last hidden layer of the first neural network that is provided immediately before an output layer of the first neural network;
- generate a second neural network for identifying a second speaker with respect to a second voice signal in a second environment in which a second SNR value of the second voice signal is less than the threshold value, wherein the second neural network is configured to set sentence frames by distinguishing a voice part of the second speaker and a silence part in which only noise exists without voice in the second voice signal, and receive the sentence frames of the second voice signal as an input of the second neural network, and identify the second speaker based on a second sentence embedding vector indicating a weighted sum of second embedding vectors that are output from a last hidden layer of the second neural network that is provided immediately before an output layer of the second neural network;
- and
- generate the speaker identification neural network by training the second neural network based on a teacher-student training model in which the first neural network is set to a teacher neural network and the second neural network is set to a student neural network,
- wherein the speaker identification neural network comprises an attention layer to adjust the initial weights of the second neural network such that a relatively high weight is assigned to an embedding vector of a period in which a voice signal exists, and a relative low weight is assigned to an embedding vector of a period in which a noise signal exists and any voice signal does not exist, among the second embedding vectors.
An apparatus for identifying a speaker with respect to an input voice signal by utilizing a speaker identification neural network, the apparatus comprising:
- a memory storing at least one program;
- and
- a processor configured to:
- extract a feature from the input voice signal;
- input the feature to the speaker identification neural network that is generated by training a second neural network according to a teacher-student training model in which a first neural network is set to a teacher neural network and the second neural network is set to a student neural network,
- wherein the first neural network is trained to set sentence frames by distinguishing a voice part of the first speaker and a silence part in which only noise exists without voice in the first voice signal, and receive the sentence frames of the first voice signal as an input of the first neural network, and identify a first speaker with respect to a first voice signal in a first environment in which a first signal-to-noise (SNR) value of the first voice signal is greater than or equal to a threshold value, based on a first sentence embedding vector representing a weighted sum of first embedding vectors that are output from a last hidden layer of the first neural network that is provided immediately before an output layer of the first neural network;
- and
- wherein the second neural network is trained to set sentence frames by distinguishing a voice part of the second speaker and a silence part in which only noise exists without voice in the second voice signal, and receive the sentence frames of the second voice signal as an input of the second neural network, and identify a second speaker with respect to a second voice signal in a second environment in which a second SNR value of the second voice signal is less than the threshold value, based on a second sentence embedding vector indicating a weighted sum of second embedding vectors that are output from a last hidden layer of the second neural network that is provided immediately before an output layer of the second neural network;
- extract the second sentence embedding vector corresponding to the feature from the speaker identification neural network;
- and
- identify the speaker with respect to the input voice signal based on the second sentence embedding vector,
- wherein the speaker identification neural network comprises an attention layer to adjust the initial weights of the second neural network such that a relatively high weight is assigned to an embedding vector of a period in which a voice signal exists, and a relative low weight is assigned to an embedding vector of a period in which a noise signal exists and any voice signal does not exist, among the second embedding vectors.
Dependencies are inferred from the claim text.