Systems and methods for using contextual interim responses in conversations managed by a virtual assistant server

A virtual assistant server comprising:

• one or more processors;
• and
• a memory coupled to the one or more processors which are configured to execute programmed instructions stored in the memory to:
• execute a dialog flow comprising a series of interconnected nodes including at least one of:
• an intent node, one or more entity nodes, one or more invoke-LLM nodes, one or more service nodes, one or more confirmation nodes, or one or more message nodes associated with a use case determined from a user input received from a user device associated with a user as part of an online conversation between the user and a virtual assistant to generate an actual response to the user input;
• provide a prompt to a large language model (LLM) comprising:
• one or more instructions to the LLM to generate a plurality of interim responses to be transmitted to the user device, wherein the plurality of interim responses are generated by the LLM using a probabilistic approach to language generation based on patterns learned from training data comprising input-output pairs of prompts and expected interim responses;
• and
• input data required by the LLM to generate the plurality of interim responses, wherein the input data comprises:
• a description of the use case, an identification of a current node of execution of the dialog flow, and a description of an action performed at the current node of execution;
• transmit one or more of the plurality of interim responses received from the LLM to the user device in an order determined by the LLM prior to generating the actual response;
• and
• transmit the generated actual response to the user input to the user device subsequent to the transmitting the one or more of the plurality of interim responses to the user device.

A non-transitory computer-readable medium storing instructions which when executed by one or more processors, causes the one or more processors to:

• execute a dialog flow comprising a series of interconnected nodes including at least one of:
• an intent node, one or more entity nodes, one or more invoke-LLM nodes, one or more service nodes, one or more confirmation nodes, or one or more message nodes associated with a use case determined from a user input received from a user device associated with a user as part of an online conversation between the user and a virtual assistant to generate an actual response to the user input;
• provide a prompt to a large language model (LLM) comprising:
• one or more instructions to the LLM to generate a plurality of interim responses to be transmitted to the user device, wherein the plurality of interim responses are generated by the LLM using a probabilistic approach to language generation based on patterns learned from training data comprising input-output pairs of prompts and expected interim responses;
• and
• input data required by the LLM to generate the plurality of interim responses, wherein the input data comprises:
• a description of the use case, an identification of a current node of execution of the dialog flow, and a description of an action performed at the current node of execution;
• transmit one or more of the plurality of interim responses received from the LLM to the user device in an order determined by the LLM prior to generating the actual response;
• and
• transmit the generated actual response to the user input to the user device subsequent to the transmitting the one or more of the plurality of interim responses to the user device.