Generation and utilization of pseudo-correction(s) to prevent forgetting of personalized on-device automatic speech recognition (ASR) model(s)

A method implemented by one or more processors of a client device, the method comprising:

• receiving, via one or more microphones of the client device, audio data that captures a spoken utterance of a user of the client device;
• processing, using a baseline instance of an on-device automatic speech recognition (ASR) model that is stored locally in on-device storage of the client device, the audio data to generate a plurality of speech hypotheses that are each predicted to correspond to the spoken utterance;
• causing a given speech hypothesis, from among the plurality of speech hypotheses, to be visually rendered for presentation to the user at a display of the client device;
• receiving, responsive to the given speech hypothesis being visually rendered, further user interface input that modifies at least a portion of the given speech hypothesis to an alternate speech hypothesis;
• in response to receiving the further user interface input:
• storing, in the on-device storage of the client device and in association with a given time to live (TTL) in the on-device storage of the client device, at least the portion of the given speech hypothesis and the alternate speech hypothesis as a correction for the baseline instance of the on-device ASR model;
• causing the baseline instance of the on-device ASR model to be updated based on at least the correction to generate a personalized instance of the on-device ASR model;
• and
• storing, in the on-device storage of the client device and in addition to the baseline instance of the on-device ASR model, the personalized instance of the on-device ASR model;
• receiving, via one or more of the microphones of the client device, additional audio data that captures an additional spoken utterance of the user;
• processing, using the baseline instance of the on-device ASR model, the additional audio data to generate a plurality of additional speech hypotheses that are each predicted to correspond to the additional spoken utterance;
• processing, using the personalized instance of the on-device ASR model, the additional audio data to generate a plurality of personalized speech hypotheses that are each predicted to correspond to the additional spoken utterance;
• determining, based on comparing one or more of the plurality of additional speech hypotheses that are each predicted to correspond to the additional spoken utterance to one or more of the plurality of personalized speech hypotheses that are each predicted to correspond to the additional spoken utterance, whether to generate a pseudo-correction that corresponds to the correction;
• and
• in response to determining to generate the pseudo-correction that corresponds to the correction:
• storing, in the on-device storage of the client device and in association with an additional given TTL in the on-device storage of the client device, at least the portion of the given speech hypothesis and the alternate speech hypothesis as a pseudo-correction for the baseline instance of on-device ASR model;
• causing the baseline instance of the on-device ASR model to be updated based on at least the pseudo-correction to generate an additional personalized instance of the on-device ASR model;
• and
• storing, in the on-device storage of the client device and in addition to the baseline instance of the on-device ASR model, the additional personalized instance of the on-device ASR model.

A method implemented by one or more processors of a client device, the method comprising:

• receiving, via one or more microphones of the client device, audio data that captures a spoken utterance of a user of the client device;
• processing, using an on-device automatic speech recognition (ASR) model that is stored locally in on-device storage of the client device, the audio data to generate a plurality of speech hypotheses that are each predicted to correspond to the spoken utterance;
• causing a given speech hypothesis, from among the plurality of speech hypotheses, to be visually rendered for presentation to the user at a display of the client device;
• receiving, responsive to the given speech hypothesis being visually rendered, further user interface input that modifies at least a portion of the given speech hypothesis to an alternate speech hypothesis;
• in response to receiving the further user interface input:
• storing, in the on-device storage of the client device and in association with a given time to live (TTL) in the on-device storage of the client device, at least the portion of the given speech hypothesis and the alternate speech hypothesis as a correction for the on-device ASR model;
• and
• causing the on-device ASR model to be updated based on at least the correction;
• receiving, via one or more of the microphones of the client device, additional audio data that captures an additional spoken utterance of the user;
• processing, using the on-device ASR model, the additional audio data to generate a plurality of additional speech hypotheses that are each predicted to correspond to the additional spoken utterance;
• and
• in response to determining that the plurality of additional speech hypotheses include at least the portion of the given speech hypothesis and the alternate speech hypothesis from the correction:
• storing, in the on-device storage of the client device and in association with an additional given TTL in the on-device storage of the client device, at least the portion of the given speech hypothesis and the alternate speech hypothesis as a pseudo-correction for the on-device ASR model;
• and
• causing the on-device ASR model to be updated based on at least the pseudo-correction.