Large language model handling out-of-scope and out-of-domain detection for digital assistant

A system comprising:

• one or more processors;
• and
• one or more computer-readable media storing instructions which, when executed by the one or more processors, cause the system to perform operations comprising:
• routing an utterance to a skill bot, wherein the skill bot is configured to execute an action for completing a task associated with the utterance, and a workflow associated with the action includes a generative artificial intelligence (GenAI) component state configured to facilitate completion of at least part of the task;
• generating, by the GenAI component state, a prompt to include the utterance and one or more scope-related elements based on a prompt template, wherein the one or more scope-related elements include:
• (i) one or more scenarios, (ii) one or more negative few-shot examples that are considered out of scope (OOS) or out of domain (OOD), and (iii) instructions that teach a GenAI model to output an invalid input variable when the GenAI model determines an utterance is OOS or OOD;
• communicating, by the GenAI component state, the prompt to a GenAI provider for processing by the GenAI model;
• receiving, at the GenAI component state from the GenAI model provider, a response generated by the GenAI model processing the prompt, wherein when the GenAI model determines the utterance is OOS or OOD as part of the processing the prompt, the response includes the invalid input variable;
• and
• responsive to the response including the invalid input variable, transitioning from the GenAI component state to another state different from that of the GenAI component state or another workflow different from the workflow associated with the action.

One or more non-transitory computer-readable media storing instructions which, when executed by one or more processors, cause the one or more processors to perform operations comprising:

• routing an utterance to a skill bot, wherein the skill bot is configured to execute an action for completing a task associated with the utterance, and a workflow associated with the action includes a generative artificial intelligence (GenAI) component state configured to facilitate completion of at least part of the task;
• generating, by the GenAI component state, a prompt to include the utterance and one or more scope-related elements based on a prompt template, wherein the one or more scope-related elements include:
• (i) one or more scenarios, (ii) one or more negative few-shot examples that are considered out of scope (OOS) or out of domain (OOD), and (iii) instructions that teach a GenAI model to output an invalid input variable when the GenAI model determines an utterance is OOS or OOD;
• communicating, by the GenAI component state, the prompt to a GenAI provider for processing by the GenAI model;
• receiving, at the GenAI component state from the GenAI model provider, a response generated by the GenAI model processing the prompt, wherein when the GenAI model determines the utterance is OOS or OOD as part of the processing the prompt, the response includes the invalid input variable;
• and
• responsive to the response including the invalid input variable, transitioning from the GenAI component state to another state different from that of the GenAI component state or another workflow different from the workflow associated with the action.