On-device text-to-speech model personalization

A method comprising:

• obtaining, by one or more processors of a device during normal operation of the device, one or more audio signals that include user speech;
• and
• performing, by the one or more processors, a sequence of sample criteria checks on speech samples associated with the one or more audio signals, wherein the sequence of sample criteria checks include:
• a check whether a confidence value associated with an automatic speech recognition (ASR) transcription of a sample exceeds a transcription confidence threshold;
• and
• a check whether a loss value associated with a personalized text-to-speech (TTS) output of the sample exceeds a loss threshold, the ASR transcription satisfies a lexicon diversity criterion, or both.

A non-transitory, computer-readable medium storing instructions that are executable by one or more processors to cause the one or more processors to:

• obtain, during normal operation of a device, one or more audio signals that include user speech;
• and
• perform a sequence of sample criteria checks on speech samples associated with the one or more audio signals, wherein the sequence of sample criteria checks include:
• a check whether a confidence value associated with an automatic speech recognition (ASR) transcription of a sample exceeds a transcription confidence threshold;
• and
• a check whether a loss value associated with a personalized text-to-speech (TTS) output of the sample exceeds a loss threshold, the ASR transcription satisfies a lexicon diversity criterion, or both.