Context-based utterance prediction for assistant systems

One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

• receive, from an assistant xbot associated with a client system associated with a first user, an initial portion of an audio input, wherein the initial portion of the audio input comprises a partial request, and wherein the initial portion is received while the first user is continuing to provide further audio input to the audio input;
• generate, by one or more computing systems responsive to receiving the initial portion of the audio input, one or more speculative queries based on the partial request and a machine-learning predictive model, wherein each speculative query is a predicted complete request based on the partial request;
• calculate, by the one or more computing systems, a confidence score for each speculative query based on the predictive model, wherein the confidence score represents a likelihood that the predicted complete request corresponding to the respective speculative query will match an intended complete request associated with the audio input after the further input is provided;
• rank, by the one or more computing systems, the one or more speculative queries based on their respective confidence scores and associated costs;
• determine a current processing load on the one or more computing systems;
• determine a threshold rank based on the current processing load, wherein the threshold rank is used to execute the speculative queries;
• execute, by the one or more computing systems, one or more of the speculative queries based on their ranks, wherein each executed speculative query has a rank greater than or equal to the threshold rank;
• cache, by the one or more computing systems, one or more results of the executed one or more speculative queries;
• and
• send, to the client system responsive to determining the first user has completed the audio input, instructions to present, to the first user by the assistant xbot, the cached results corresponding to an executed speculative query matching the completed audio input.

A system comprising:

• one or more processors;
• and a non-transitory memory coupled to the processors comprising instructions executable by the processors, the processors operable when executing the instructions to:
• receive, from an assistant xbot associated with a client system associated with a first user, an initial portion of an audio input, wherein the initial portion of the audio input comprises a partial request, and wherein the initial portion is received while the first user is continuing to provide further audio input to the audio input;
• generate, by one or more computing systems responsive to receiving the initial portion of the audio input, one or more speculative queries based on the partial request and a machine-learning predictive model, wherein each speculative query is a predicted complete request based on the partial request;
• calculate, by the one or more computing systems, a confidence score for each speculative query based on the predictive model, wherein the confidence score represents a likelihood that the predicted complete request corresponding to the respective speculative query will match an intended complete request associated with the audio input after the further input is provided;
• rank, by the one or more computing systems, the one or more speculative queries based on their respective confidence scores and associated costs;
• determine a current processing load on the one or more computing systems;
• determine a threshold rank based on the current processing load, wherein the threshold rank is used to execute the speculative queries;
• execute, by the one or more computing systems, one or more of the speculative queries based on their ranks, wherein each executed speculative query has a rank greater than or equal to the threshold rank;
• cache, by the one or more computing systems, one or more results of the executed one or more speculative queries;
• and
• send, to the client system responsive to determining the first user has completed the audio input, instructions to present, to the first user by the assistant xbot, the cached results corresponding to an executed speculative query matching the completed audio input.