Speaker awareness using speaker dependent speech model(s)

A client device including one or more processors that execute instructions, stored in memory of the client device, to perform a method comprising:

• training a speaker dependent speech model (SD speech model) based on a trained speaker independent speech model (SI speech model), wherein the SD speech model is personalizable to any user of a corresponding client device, and wherein training the SD speech model comprises:
• identifying an instance of audio data that captures one or more spoken utterances of a target user;
• determining a speaker embedding for the target user, the speaker embedding being generated based on audio data from the target user;
• generating a noisy instance of audio data by combining the instance of audio data with one or more additional sounds that are not from the target user;
• processing the instance of audio data using the SI speech model to generate speaker independent output (SI output);
• processing the noisy instance of the audio data, along with the speaker embedding for the target user, using the SD speech model to generate speaker dependent output (SD output);
• generating a loss based on the SI output and the SD output;
• and
• updating one or more portions of the SD model based on the generated loss.

A non-transitory computer-readable storage medium storing instructions executable by one or more processors of a computing system that, when executed, cause the one or more processors to perform operations comprising:

• training a speaker dependent speech model (SD speech model) based on a trained speaker independent speech model (SI speech model), wherein the SD speech model is personalizable to any user of a corresponding client device, and wherein training the SD speech model comprises:
• identifying an instance of audio data that captures one or more spoken utterances of a target user;
• determining a speaker embedding for the target user, the speaker embedding being generated based on audio data from the target user;
• generating a noisy instance of audio data by combining the instance of audio data with one or more additional sounds that are not from the target user;
• processing the instance of audio data using the SI speech model to generate speaker independent output (SI output);
• processing the noisy instance of the audio data, along with the speaker embedding for the target user, using the SD speech model to generate speaker dependent output (SD output);
• generating a loss based on the SI output and the SD output;
• and
• updating one or more portions of the SD model based on the generated loss.