# US 10,121,476 B2 · System and method for generating challenge utterances for speaker verification > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,121,476 B2 - Document ID: US10121476B2 - Status: Granted - Kind: B2 - Application number: 15/075,333 - Filed: 2016-03-21 - Published: 2016-07-14 - Granted: 2018-11-06 - Earliest claimed: 2010-11-24 - Assignees: Nuance Communications, Inc. - Applicants: Nuance Communications, Inc. - Inventors: Ilija Zeljkovic; Taniya Mishra; Amanda Stent; Ann K. Syrdal; Jay Wilpon - Web page: https://patentgrep.com/patent/US10121476B2 ## CPC - G10L 17/24: the user being prompted to utter a password or a predefined phrase - G10L 15/02: Feature extraction for speech recognition; Selection of recognition unit - G10L 17/04: Training, enrolment or model building - G10L 15/08: Speech classification or search - G10L 17/00: Speaker identification or verification techniques - G10L 17/26: Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices - G10L 2015/025: {Phonemes, fenemes or fenones being the recognition units} - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract Disclosed herein are systems, methods, and non-transitory computer-readable storage media relating to speaker verification. In one aspect, a system receives a first user identity from a second user, and, based on the identity, accesses voice characteristics. The system randomly generates a challenge sentence according to a rule and/or grammar, based on the voice characteristics, and prompts the second user to speak the challenge sentence. The system verifies that the second user is the first user if the spoken challenge sentence matches the voice characteristics. In an enrollment aspect, the system constructs an enrollment phrase that covers a minimum threshold of unique speech sounds based on speaker-distinctive phonemes, phoneme clusters, and prosody. Then user utters the enrollment phrase and extracts voice characteristics for the user from the uttered enrollment phrase. The system generates a user profile, based on the voice characteristics, for generating random challenge sentences according to a grammar. ## Claims ### Claim 1 A method comprising: collecting user speech uttered while a user is using a speech service associated with a multi-platform automatic speech recognition engine to yield collected user speech; identifying speech of interest from the collected user speech, wherein the speech of interest has high speaker discriminatory ability; identifying, from the speech of interest, voice characteristics for the user, the voice characteristics comprising one or more of speaker-distinctive phonemes, phoneme clusters, and prosody, to yield identified voice characteristics; creating a user profile for the user by flagging the identified voice characteristics in the user profile as important for generating challenge sentences to yield flagged voice characteristics; generating, by a system comprising a processor and based on the flagged voice characteristics in the user profile, a random challenge sentence according to a challenge grammar; and prompting the user to utter the random challenge sentence to verify an identity of the user. ### Claim 2 (depends on 1) The method of claim 1, further comprising: collecting additional user speech; identifying additional voice characteristics from the additional user speech; and updating the user profile based on the additional voice characteristics. ### Claim 3 (depends on 1) The method of claim 1, wherein the voice characteristics are associated with different intonation patterns. ### Claim 4 (depends on 1) The method of claim 1, wherein the random challenge sentence is generated according to one of a plurality of sentence frames. ### Claim 5 (depends on 1) The method of claim 1, further comprising adding, by the system, personal information associated with the user to the user profile. ### Claim 6 (depends on 1) The method of claim 1, further comprising adding, by the system, personal contacts associated with the user to the user profile. ### Claim 7 A non-transitory computer-readable storage medium storing instructions which, responsive to being executed by a processor, cause the processor to perform operations comprising: collecting user speech uttered while a user is using a speech service associated with a multi-platform automatic speech recognition engine to yield collected user speech; identifying speech of interest from the collected user speech, wherein the speech of interest has high speaker discriminatory ability; identifying, from the speech of interest, voice characteristics for the user, the voice characteristics comprising one or more of speaker-distinctive phonemes, phoneme clusters, and prosody, to yield identified voice characteristics; creating a user profile for the user by flagging the identified voice characteristics in the user profile as important for generating challenge sentences to yield flagged voice characteristics; generating, based on the flagged voice characteristics of interest in the user profile, a random challenge sentence according to a challenge grammar; and prompting the user to utter the random challenge sentence to verify an identity of the user. ### Claim 8 (depends on 7) The non-transitory computer-readable storage medium of claim 7, wherein the voice characteristics are associated with different intonation patterns. ### Claim 9 (depends on 7) The non-transitory computer-readable storage medium of claim 7, wherein the random challenge sentence is generated according to one of a plurality of sentence frames. ### Claim 10 A system comprising: a processor; and a computer-readable storage medium having instructions stored which, responsive to being executed by the processor, result in the processor performing operations comprising: collecting user speech uttered while a user is using a speech service associated with a multi-platform automatic speech recognition engine to yield collected user speech; identifying speech of interest from the collected user speech, wherein the speech of interest has high speaker discriminatory ability; identifying, from the speech of interest, voice characteristics for the user, the voice characteristics comprising one or more of speaker-distinctive phonemes, phoneme clusters, and prosody, to yield identified voice characteristics; creating a user profile for the user by flagging the identified voice characteristics in the user profile as important for generating challenge sentences to yield flagged voice characteristics; generating, based on the flagged voice characteristics in the user profile, a random challenge sentence according to a challenge grammar; and prompting the user to utter the random challenge sentence to verify an identity of the user. ### Claim 11 (depends on 10) The system of claim 10, wherein the operations further comprise: collecting additional user speech; identifying additional voice characteristics from the additional user speech; and updating the user profile based on the additional voice characteristics. ### Claim 12 (depends on 10) The system of claim 10, wherein the voice characteristics are associated with different intonation patterns. ### Claim 13 (depends on 10) The system of claim 10, wherein the random challenge sentence is generated according to one of a plurality of sentence frames. ### Claim 14 (depends on 10) The system of claim 10, wherein the operations further comprise adding personal information associated with the user to the user profile. ### Claim 15 (depends on 10) The system of claim 10, wherein the operations further comprise adding personal contacts associated with the user to the user profile. ### Claim 16 (depends on 10) The system of claim 10, wherein the generating of the random challenge sentence comprises: identifying, using the voice characteristics, a plurality of more valuable phonemes determined to be more valuable than second phonemes for verifying an asserted identity; and generating the random challenge sentence, based on the voice characteristics, wherein the random challenge sentence comprises the plurality of more valuable phonemes.