Proactive assistance via a cascade of LLMS

A system comprising:

• data processing hardware;
• and
• memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising:
• obtaining, by a digital assistant, a contextual event associated with a user of a user device;
• generating, using a local large language model (LLM) executing on the user device, initial response content associated with the contextual event, the initial response content providing an offer for the digital assistant to interact with a remote LLM to perform an action on the user's behalf based on the contextual event;
• providing, for output from the user device, initial presentation content based on the initial response content, the initial presentation content prompting the user to consent to the offer for the digital assistant to interact with the remote LLM to perform the action on the user's behalf;
• receiving an initial presentation content interaction indicating user interaction with the initial presentation content;
• determining, using the local LLM, a remote LLM prompt confidence based on the received initial presentation content, the remote LLM prompt confidence indicating a likelihood of prompting a remote LLM for proactive assistance associated with the contextual event;
• determining that the remote LLM prompt confidence satisfies a threshold;
• based on determining that the remote LLM prompt confidence satisfies the threshold, generating a remote LLM prompt for the remote LLM executing remote from the user device;
• transmitting, to the remote LLM, the remote LLM prompt;
• receiving, at the digital assistant, from the remote LLM, response content providing the proactive assistance associated with the contextual event;
• and
• providing, for output from the user device, presentation content based on the response content received from the remote LLM.