Large language model-predicted response biasing for conversational systems

A system comprising:

• data processing hardware;
• and
• memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising:
• during an assistant turn in a voice-based conversation between a user and an assistant large language model (LLM), receiving a response generated by the assistant LLM that is directed toward the user;
• based on the response generated by the assistant LLM that is directed toward the user, predicting, by the assistant LLM, one or more possible terms the user may speak in a follow-up query to the response generated by the assistant LLM during a user turn subsequent to the assistant turn in the voice-based conversation;
• biasing an automated speech recognition (ASR) model toward recognizing the one or more possible terms predicted by the assistant LLM;
• during the user turn subsequent to the assistant turn in the voice-based conversation, receiving audio data characterizing the follow-up query to the response generated by the assistant LLM;
• and
• processing, using the ASR model biased toward recognizing the one or more possible terms predicted by the assistant LLM, the audio data to generate a transcription of the follow-up query.