# US 2014/0195236 A1 · Speaker verification and identification using artificial neural network-based sub-phonetic unit discrimination > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0195236 A1 - Document ID: US20140195236A1 - Status: Published application - Kind: A1 - Application number: 13/738,868 - Filed: 2013-01-10 - Published: 2014-07-10 - Earliest claimed: 2013-01-10 - Assignees: Sensory, Incorporated - Inventors: John-Paul Hosom; Pieter J. Vermeulen; Jonathan Shaw - Web page: https://patentgrep.com/patent/US20140195236A1 ## CPC - G10L 17/14: Use of phonemic categorisation or speech recognition prior to speaker recognition or verification - G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections - G06N 3/084: Backpropagation, e.g. using gradient descent - G10L 17/04: Training, enrolment or model building - G10L 17/18: Artificial neural networks; Connectionist approaches - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract In one embodiment, a computer system stores speech data for a plurality of speakers, where the speech data includes a plurality of feature vectors and, for each feature vector, an associated sub-phonetic class. The computer system then builds, based on the speech data, an artificial neural network (ANN) for modeling speech of a target speaker in the plurality of speakers, where the ANN is configured to discriminate between instances of sub-phonetic classes uttered by the target speaker and instances of sub-phonetic classes uttered by other speakers in the plurality of speakers. ## Claims ### Claim 1 A method comprising: storing, by a computer system, speech data for a plurality of speakers, the speech data including a plurality of feature vectors and, for each feature vector, an associated sub-phonetic class; and building, by the computer system based on the speech data, an artificial neural network (ANN) for modeling speech of a target speaker in the plurality of speakers, the ANN being configured to discriminate between instances of sub-phonetic classes uttered by the target speaker and instances of sub-phonetic classes uttered by other speakers in the plurality of speakers. ### Claim 2 (depends on 1) The method of claim 1 wherein building the ANN comprises: retrieving an existing ANN that comprises a plurality of existing output nodes, each existing output node corresponding to a sub-phonetic class and being configured to output a probability that a feature vector input to the existing ANN is an instance of the sub-phonetic class uttered by one of the other speakers in the plurality of speakers; and modifying the existing ANN to generate the ANN, wherein the modifying causes the ANN to include an output layer that comprises the plurality of existing output nodes and, for each existing output node, a new output node that corresponds to the sub-phonetic class of the existing output node, the new output node being configured to output a probability that a feature vector input to the ANN is an instance of the sub-phonetic class uttered by the target speaker. ### Claim 3 (depends on 2) The method of claim 2 wherein building the ANN further comprises: training the ANN using a first portion of the speech data originating from the target speaker such that, for each feature vector and associated sub-phonetic class in the first portion, the new output node that corresponds to the associated sub-phonetic class is tuned to output a relatively higher probability and the existing output node that corresponds to the associated sub-phonetic class is tuned to output a relatively lower probability. ### Claim 4 (depends on 3) The method of claim 3 wherein building the ANN further comprises: training the ANN using a second portion of the speech data originating from the other speakers such that, for each feature vector and associated sub-phonetic class in the second portion, the new output node corresponding to the associated sub-phonetic class is tuned to output a relatively lower probability and the existing output node corresponding to the associated sub-phonetic class is tuned to output a relatively higher probability. ### Claim 5 (depends on 4) The method of claim 4 wherein the ANN further includes an input layer comprising a plurality of input nodes and one or more hidden layers comprising one or more hidden nodes. ### Claim 6 (depends on 5) The method of claim 5 wherein the input layer is connected to a lowest hidden layer in the one or more hidden layers via a first set of connections having a first set of weights, wherein the one or more hidden layers are connected from a lower layer to a higher layer via a second set of connections having a second set of weights, wherein a highest hidden layer in the one or more hidden layers is connected to the output layer via a third set of connections having a third set of weights, and wherein training the ANN using the first and second portions of the speech data comprises applying a back-propagation algorithm to modify the third set of weights without modifying the first or second set of weights. ### Claim 7 (depends on 1) The method of claim 1 further comprising: receiving an acoustic signal corresponding to an utterance of a target phrase; extracting feature vectors from a plurality of frames in the acoustic signal; and generating, based on the feature vectors and the ANN, a speaker-verification score indicating a likelihood that the target phrase was uttered by the target speaker. ### Claim 8 (depends on 7) The method of claim 7 wherein generating the speaker-verification score comprises performing a Viterbi search using a Hidden Markov Model (HMM) that lexically models the target phrase. ### Claim 9 (depends on 8) The method of claim 8 wherein, at each frame in the plurality of frames, the Viterbi search passes a feature vector corresponding to the frame as input to the ANN and generates a per-frame speaker-verification score based on one or more probabilities output by the ANN. ### Claim 10 (depends on 9) The method of claim 9 wherein the per-frame speaker-verification score for a state in the HMM corresponds to the probability output by the new output node in the ANN which is associated with that state in the HMM. ### Claim 11 (depends on 9) The method of claim 9 wherein the per-frame speaker-verification score for a state in the HMM corresponds to the probability output by a new output node in the ANN which is associated with that state in the HMM, less the probability output by the existing output node associated with the new output node in the ANN. ### Claim 12 (depends on 9) The method of claim 9 wherein generating the speaker-verification score further comprises applying a function to one or more of the per-frame speaker-verification scores to calculate the speaker-verification score. ### Claim 13 (depends on 12) The method of claim 12 wherein the function is an average function or a summation function. ### Claim 14 (depends on 12) The method of claim 12 wherein the function is applied to a subset of the per-frame speaker-verification scores that correspond to relevant frames in the plurality of frames, each relevant frame being classified, via the Viterbi search, as being part of a phoneme that is considered useful for speaker verification or identification. ### Claim 15 A non-transitory computer readable storage medium having stored thereon program code executable by a computer system, the program code comprising: code that causes the computer system to store speech data for a plurality of speakers, the speech data including a plurality of feature vectors and, for each feature vector, an associated sub-phonetic class; and code that causes the computer system to build, based on the speech data, an artificial neural network (ANN) for modeling speech of a target speaker in the plurality of speakers, the ANN being configured to discriminate between instances of sub-phonetic classes uttered by the target speaker and instances of sub-phonetic classes uttered by other speakers in the plurality of speakers. ### Claim 16 (depends on 15) The non-transitory computer readable storage medium of claim 15 wherein the code that causes the computer system to build the ANN comprises: code that causes the computer system to retrieve an existing ANN that comprises a plurality of existing output nodes, each existing output node corresponding to a sub-phonetic class and being configured to output a probability that a feature vector input to the existing ANN is an instance of the sub-phonetic class uttered by one of the other speakers in the plurality of speakers; and code that causes the computer system to modify the existing ANN to generate the ANN, wherein the modifying causes the ANN to include an output layer that comprises the plurality of existing output nodes and, for each existing output node, a new output node that corresponds to the sub-phonetic class of the existing output node, the new output node being configured to output a probability that a feature vector input to the ANN is an instance of the sub-phonetic class uttered by the target speaker. ### Claim 17 (depends on 16) The non-transitory computer readable storage medium of claim 16 wherein the code that causes the computer system to build the ANN further comprises: code that causes the computer system to train the ANN using a first portion of the speech data originating from the target speaker such that, for each feature vector and associated sub-phonetic class in the first portion, the new output node that corresponds to the associated sub-phonetic class is tuned to output a relatively higher probability and the existing output node that corresponds to the associated sub-phonetic class is tuned to output a relatively lower probability; and code that causes the computer system to train the ANN using a second portion of the speech data originating from the other speakers such that, for each feature vector and associated sub-phonetic class in the second portion, the new output node corresponding to the associated sub-phonetic class is tuned to output a relatively lower probability and the existing output node corresponding to the associated sub-phonetic class is tuned to output a relatively higher probability. ### Claim 18 A system comprising: a processor configured to: store speech data for a plurality of speakers, the speech data including a plurality of feature vectors and, for each feature vector, an associated sub-phonetic class; and build, based on the speech data, an artificial neural network (ANN) for modeling speech of a target speaker in the plurality of speakers, the ANN being configured to discriminate between instances of sub-phonetic classes uttered by the target speaker and instances of sub-phonetic classes uttered by other speakers in the plurality of speakers. ### Claim 19 (depends on 18) The system of claim 18 wherein building the ANN comprises: retrieving an existing ANN that comprises a plurality of existing output nodes, each existing output node corresponding to a sub-phonetic class and being configured to output a probability that a feature vector input to the existing ANN is an instance of the sub-phonetic class uttered by one of the other speakers in the plurality of speakers; and modifying the existing ANN to generate the ANN, wherein the modifying causes the ANN to include an output layer that comprises the plurality of existing output nodes and, for each existing output node, a new output node that corresponds to the sub-phonetic class of the existing output node, the new output node being configured to output a probability that a feature vector input to the ANN is an instance of the sub-phonetic class uttered by the target speaker. ### Claim 20 (depends on 19) The system of claim 19 wherein building the ANN further comprises: training the ANN using a first portion of the speech data originating from the target speaker such that, for each feature vector and associated sub-phonetic class in the first portion, the new output node that corresponds to the associated sub-phonetic class is tuned to output a relatively higher probability and the existing output node that corresponds to the associated sub-phonetic class is tuned to output a relatively lower probability; and training the ANN using a second portion of the speech data originating from the other speakers such that, for each feature vector and associated sub-phonetic class in the second portion, the new output node corresponding to the associated sub-phonetic class is tuned to output a relatively lower probability and the existing output node corresponding to the associated sub-phonetic class is tuned to output a relatively higher probability.