# US 2020/0286491 A1 · Speaker verification based on a speaker template > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2020/0286491 A1 - Document ID: US20200286491A1 - Status: Published application - Kind: A1 - Application number: 16/296,733 - Filed: 2019-03-08 - Published: 2020-09-10 - Earliest claimed: 2019-03-08 - Assignees: QUALCOMM Incorporated - Applicants: QUALCOMM Incorporated - Inventors: Sunkuk MOON; Bicheng JIANG; Erik VISSER - Web page: https://patentgrep.com/patent/US20200286491A1 ## CPC - G10L 17/22: Interactive procedures; Man-machine interfaces - G10L 17/00: Speaker identification or verification techniques - G10L 17/02: Preprocessing operations, e.g. segment selection; Pattern representation or modelling, e.g. based on linear discriminant analysis [LDA] or principal components; Feature selection or extraction - G10L 17/04: Training, enrolment or model building - G10L 17/06: Decision making techniques; Pattern matching strategies - G10L 17/08: Use of distortion metrics or a particular distance between probe pattern and reference templates - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract A device includes a processor configured to determine a feature vector based on an utterance and to determine a first embedding vector by processing the feature vector using a trained embedding network. The processor is configured to determine a first distance metric based on distances between the first embedding vector and each embedding vector of a speaker template. The processor is configured to determine, based on the first distance metric, that the utterance is verified to be from a particular user. The processor is configured to, based on a comparison of a first particular distance metric associated with the first embedding vector to a second distance metric associated with a first test embedding vector of the speaker template, generate an updated speaker template by adding the first embedding vector as a second test embedding vector and removing the first test embedding vector from test embedding vectors of the speaker template. ## Claims ### Claim 1 A device comprising: a memory configured to store: a trained embedding network; and a speaker template associated with a particular user, the speaker template including one or more enrollment embedding vectors based on initial user enrollment data and including one or more test embedding vectors; and a processor coupled to the memory, the processor configured to: determine a first feature vector based on a first utterance; determine a first embedding vector based on the first feature vector by processing the first feature vector using the trained embedding network; determine a first distance metric based on distances between the first embedding vector and each embedding vector of the speaker template; perform a speaker verification operation to determine, based on the first distance metric, whether the first utterance is verified to be from the particular user; based on determining that the first utterance is verified to be from the particular user, perform a comparison of a first particular distance metric associated with the first embedding vector to a second distance metric associated with a first test embedding vector of the speaker template; and based on the comparison, generate an updated speaker template by adding the first embedding vector as a second test embedding vector and removing the first test embedding vector from the test embedding vectors of the speaker template. ### Claim 2 (depends on 1) The device of claim 1, wherein the processor is configured to determine that the first utterance is verified to be from the particular user based on determining that the first distance metric satisfies a speaker verification threshold. ### Claim 3 (depends on 1) The device of claim 1, wherein the processor is further configured to determine the first particular distance metric based on second distances between the first embedding vector and each of the enrollment embedding vectors of the speaker template. ### Claim 4 (depends on 1) The device of claim 1, wherein the processor is configured to update the speaker template based on determining that the first particular distance metric is less than the second distance metric. ### Claim 5 (depends on 1) The device of claim 1, wherein the second distance metric of the first test embedding vector is highest among distance metrics associated with the test embedding vectors. ### Claim 6 (depends on 1) The device of claim 1, further comprising a microphone coupled to the processor, the microphone configured to receive the first utterance. ### Claim 7 (depends on 1) The device of claim 1, wherein the processor is configured to: generate the initial user enrollment data during an initial enrollment period; and generate the test embedding vectors based on utterances received during a verification period that is subsequent to the initial enrollment period. ### Claim 8 (depends on 1) The device of claim 1, wherein the processor is configured to, based on determining that a count of the test embedding vectors fails to satisfy a count threshold, perform the comparison of the first particular distance metric to the second distance metric. ### Claim 9 (depends on 1) The device of claim 1, wherein the processor is configured to, subsequent to generating the updated speaker template and based on determining that a model check condition is satisfied, generate a third distance metric for the second test embedding vector, the third distance metric based on distances between the second test embedding vector and each of the enrollment embedding vectors. ### Claim 10 (depends on 9) The device of claim 9, wherein the processor is configured to, based on determining that the third distance metric fails to satisfy a trusted distance threshold, generate an alert requesting re-enrollment of the particular user. ### Claim 11 (depends on 9) The device of claim 9, wherein the processor is configured to, based on determining that the third distance metric fails to satisfy a trusted distance threshold, modify the speaker template by removing the test embedding vectors from the speaker template. ### Claim 12 (depends on 9) The device of claim 9, wherein the processor is configured to determine that the model check condition is satisfied based on determining that a count of the test embedding vectors is greater than or equal to a first threshold, detecting expiration of a model check time period, determining that a count of processed utterances is greater than or equal to a second threshold, or a combination thereof. ### Claim 13 A method of speaker verification, the method comprising: determining, at a device, a first feature vector based on a first utterance; determining, at the device, a first embedding vector based on the first feature vector by processing the first feature vector using a trained embedding network; determining, at the device, a first distance metric based on distances between the first embedding vector and each embedding vector of a speaker template associated with a particular user, the speaker template including one or more enrollment embedding vectors based on initial user enrollment data and including one or more test embedding vectors; determining, at the device, that the first utterance is verified to be from the particular user based on determining that the first distance metric satisfies a speaker verification threshold; based on determining that the first utterance is verified to be from the particular user, performing a comparison of a first particular distance metric associated with the first embedding vector to a second distance metric associated with a first test embedding vector of the speaker template; and based on the comparison, generating an updated speaker template by adding the first embedding vector as a second test embedding vector and removing the first test embedding vector from the test embedding vectors of the speaker template. ### Claim 14 (depends on 13) The method of claim 13, further comprising: determining, at the device, a second feature vector based on a second utterance; determining, at the device, a second embedding vector based on the second feature vector by processing the second feature vector using the trained embedding network; determining, at the device, a third distance metric based on distances between the second embedding vector and each embedding vector of the updated speaker template; and determining, based on the third distance metric, that the second utterance is not verified to be from the particular user. ### Claim 15 (depends on 13) The method of claim 13, further comprising: determining, at the device, a second feature vector based on a second utterance; determining, at the device, a second embedding vector based on the second feature vector by processing the second feature vector using the trained embedding network; determining, at the device, a third distance metric based on distances between the second embedding vector and each embedding vector of the updated speaker template; determining, based on the third distance metric, that the second utterance is verified to be from the particular user; and based on determining that the second utterance is verified to be from the particular user and determining that a count of the test embedding vectors of the updated speaker template satisfies a count threshold, adding the second utterance as a third test embedding vector to the test embedding vectors of the updated speaker template. ### Claim 16 (depends on 13) The method of claim 13, further comprising: determining, at the device, a second feature vector based on a second utterance; determining, at the device, a second embedding vector based on the second feature vector by processing the second feature vector using the trained embedding network; determining, at the device, a third distance metric based on distances between the second embedding vector and each embedding vector of the updated speaker template; determining, based on the third distance metric, that the second utterance is verified to be from the particular user; based on determining that the second utterance is verified to be from the particular user and determining that a count of the test embedding vectors of the updated speaker template fails to satisfy a count threshold, performing a second comparison of a third particular distance metric associated with the second embedding vector to a fourth distance metric associated with a fourth test embedding vector of the updated speaker template; and based on the second comparison, refraining from adding the second embedding vector to the updated speaker template. ### Claim 17 (depends on 13) The method of claim 13, further comprising: determining training feature vectors based on utterances associated with a plurality of users; determining training embedding vectors by using an embedding network to process the training feature vectors; generating a set of triplets based on the training embedding vectors, a particular triplet including a first training embedding vector associated with a first training utterance of a first user, a second training embedding vector associated with a second training utterance of the first user, and a third training embedding vector associated with a third training utterance of a second user; determining distance metrics corresponding to the set of triplets, a first distance metric of the particular triplet based on a difference between a first distance and a second distance, wherein the first distance is between the first training embedding vector and the second training embedding vector, and wherein the second distance is between the first training embedding vector and the third training embedding vector; selecting a first subset of the set of triplets based on the distance metrics, the particular triplet selected in the first subset based on determining that the first distance metric satisfies a tolerance threshold; and generating the trained embedding network by training the embedding network using the first subset of the set of triplets prior to training the embedding network using one or more remaining subsets of the set of triplets. ### Claim 18 (depends on 17) The method of claim 17, further comprising, prior to determining the training embedding vectors, generating the embedding network by removing an output layer from a classification network. ### Claim 19 (depends on 17) The method of claim 17, further comprising, subsequent to training the embedding network using the one or more remaining subsets of the set of triplets, training the embedding network using the set of the triplets. ### Claim 20 (depends on 13) The method of claim 13, further comprising: generating, at the device, the initial user enrollment data during an initial enrollment period; and determining, at the device, the test embedding vectors based on utterances received during a verification period that is subsequent to the initial enrollment period. ### Claim 21 (depends on 13) The method of claim 13, further comprising, subsequent to generating the updated speaker template and based on determining that a model check condition is satisfied, generating a third distance metric for the second test embedding vector, the third distance metric based on distances between the second test embedding vector and each of the enrollment embedding vectors. ### Claim 22 (depends on 21) The method of claim 21, further comprising, based on determining that the third distance metric fails to satisfy a trusted distance threshold, generating an alert requesting re-enrollment of the particular user. ### Claim 23 (depends on 21) The method of claim 21, further comprising, based on determining that the third distance metric fails to satisfy a trusted distance threshold, modifying the speaker template by removing the test embedding vectors from the speaker template. ### Claim 24 (depends on 21) The method of claim 21, further comprising determining, at the device, that the model check condition is satisfied based on determining that a count of the test embedding vectors is greater than or equal to a first threshold, detecting expiration of a model check time period, determining that a count of processed utterances is greater than or equal to a second threshold, or a combination thereof. ### Claim 25 A computer-readable storage device storing instructions that, when executed by a processor, cause the processor to perform operations comprising: determining a first feature vector based on a first utterance; determining a first embedding vector based on the first feature vector by processing the first feature vector using a trained embedding network; determining a first distance metric based on distances between the first embedding vector and each embedding vector of a speaker template associated with a particular user, the speaker template including one or more enrollment embedding vectors based on initial user enrollment data and including one or more test embedding vectors; performing a speaker verification operation to determine, based on the first distance metric, whether the first utterance is verified to be from the particular user; based on determining that the first utterance is verified to be from the particular user, performing a comparison of a first particular distance metric associated with the first embedding vector to a second distance metric associated with a first test embedding vector of the speaker template; and based on the comparison, generating an updated speaker template by adding the first embedding vector as a second test embedding vector and removing the first test embedding vector from the test embedding vectors of the speaker template. ### Claim 26 (depends on 25) The computer-readable storage device of claim 25, wherein the operations further comprise determining that the first utterance is verified to be from the particular user based on determining that the first distance metric satisfies a speaker verification threshold. ### Claim 27 (depends on 25) The computer-readable storage device of claim 25, wherein the second distance metric of the first test embedding vector is highest among distance metrics associated with the test embedding vectors. ### Claim 28 (depends on 25) The computer-readable storage device of claim 25, wherein the operations further comprise: determining training feature vectors based on utterances associated with a plurality of users; determining training embedding vectors by using an embedding network to process the training feature vectors; determining distance metrics corresponding to the training embedding vectors; selecting, based on the distance metrics, a first subset of the training embedding vectors; and generating the trained embedding network by training the embedding network using the first subset of the training embedding vectors prior to training the embedding network using one or more remaining subsets of the training embedding vectors. ### Claim 29 An apparatus comprising: means for storing a trained embedding network; means for storing a speaker template associated with a particular user, the speaker template including one or more enrollment embedding vectors based on initial user enrollment data and including one or more test embedding vectors; means for determining a first feature vector based on a first utterance; means for determining a first embedding vector based on the first feature vector by processing the first feature vector using the trained embedding network; means for determining a first distance metric based on distances between the first embedding vector and each embedding vector of the speaker template; means for performing a speaker verification operation to determine, based on the first distance metric, whether the first utterance is verified to be from the particular user; means for performing a comparison of a first particular distance metric associated with the first embedding vector to a second distance metric associated with a first test embedding vector of the speaker template, the comparison performed based on determining that the first utterance is verified to be from the particular user; and means for generating an updated speaker template by adding the first embedding vector as a second test embedding vector and removing the first test embedding vector from the test embedding vectors of the speaker template, the updated speaker template generated based on the comparison. ### Claim 30 (depends on 29) The apparatus of claim 29, wherein the means for storing the trained embedding network, the means for storing the speaker template, the means for determining the first feature vector, the means for determining the first embedding vector, the means for determining the first distance metric, the means for performing the speaker verification operation, and means for performing the comparison are integrated into at least one of a virtual assistant, a home appliance, a smart device, an internet of things (IoT) device, a mobile phone, base station, a communication device, a computer, a music player, a video player, an entertainment unit, a navigation device, a personal digital assistant (PDA), a decoder, or a set top box.