Phrase extraction for ASR models

A system comprising:

• data processing hardware;
• and
• memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising:
• obtaining audio data characterizing an utterance and a corresponding ground-truth transcription of the utterance;
• processing the ground-truth transcription to identify a particular phrase included in the ground-truth transcription that is associated with sensitive data;
• modifying the audio data to obfuscate the particular phrase recited in the utterance;
• processing, using a trained automated speech recognition (ASR) model, the modified audio data to generate a predicted transcription of the utterance;
• determining whether the predicted transcription includes the particular phrase or another phrase substituted for the particular phrase from the ground-truth transcription that is associated with a same category of information as the particular phrase by comparing the predicted transcription of the utterance to the ground-truth transcription of the utterance;
• and
• when the predicted transcription includes the other phrase substituted for the particular phrase, generating an output indicating that the trained ASR model leaked the other phrase from a training data set used to train the ASR model.