System and method for speaker verification for voice assistant

An apparatus comprising:

• at least one processing device configured to:
• obtain audio data;
• identify an utterance of a wake word or phrase in the audio data;
• generate an embedding vector based on the utterance from the audio data;
• access a set of previously-generated vectors representing previous utterances of the wake word or phrase;
• perform clustering on the embedding vector and the set of previously-generated vectors to identify a cluster including the embedding vector, the identified cluster associated with a speaker;
• update a speaker vector associated with the speaker based on the embedding vector;
• determine, using a speaker verification model, a similarity score between the updated speaker vector and the embedding vector;
• and
• determine, based on the similarity score, whether a speaker providing the utterance matches the speaker associated with the identified cluster.

A non-transitory computer readable medium containing instructions that when executed cause at least one processor to:

• obtain audio data;
• identify an utterance of a wake word or phrase in the audio data;
• generate an embedding vector based on the utterance from the audio data;
• access a set of previously-generated vectors representing previous utterances of the wake word or phrase;
• perform clustering on the embedding vector and the set of previously-generated vectors to identify a cluster including the embedding vector, the identified cluster associated with a speaker;
• update a speaker vector associated with the speaker based on the embedding vector;
• determine, using a speaker verification model, a similarity score between the updated speaker vector and the embedding vector;
• and
• determine, using based on the similarity score, whether a speaker providing the utterance matches the speaker associated with the identified cluster.