Method and system for generative ai based unified virtual assistant

A system, comprising:

• a memory storing instructions;
• one or more communication interfaces;
• and
• one or more hardware processors coupled to the memory via the one or more communication interfaces, wherein the one or more hardware processors are configured by the instructions to:
• receive in real time by a virtual assistant engine of a unified virtual assistant, a multi-modal query from a user associated with a role in a user conversation using an enterprise application associated with an enterprise;
• create by the unified virtual assistant, a user context for the user conversation based on the role associated with the user, wherein the user context is a context which contains information related to a current session of chat being done by a specific user, wherein the user context enables to maintain linking of sequential and back-to-back queries to have a correlation being done in multiple queries in correct manner, wherein the user context is created by instantiating object of chat agent custom class and storing in session provided by UI library, wherein the UI library is used to create a user interface of the unified virtual assistant;
• generate by the unified virtual assistant, an optimized prompt for the multi-modal query corresponding to the user context, based on a set of prompt concepts, wherein the optimized prompt is generated by passing a required prefix or suffix as an input to an agent along with query of the user as another parameter, wherein the agent then modifies the prompt with specified prefix or suffix to get the best response, wherein the passed prefix or suffix are changed or modified dynamically based on the role of the user;
• training a large language model (LLM) using an enterprise context corresponding to the enterprise and a set of user contexts stored in a database;
• generate by the unified virtual assistant, a response corresponding to the optimized prompt from a large language model (LLM) using a customized tool array, wherein the customized tool array comprises a set of tools with each tool comprising a set of parameters including a tool description, wherein an optimal tool from the customized tool array of the virtual assistant engine is determined based on comparison of the optimal prompt to a set of parameters comprised in the customized tool array, wherein the agent using the LLM finds the optimal tool by executing the tool, halting till tool's response is received as an observation for the LLM and decides the next action based on all preceding responses;
• format by the unified virtual assistant, the response to obtain a final output using an output parser;
• and
• provide by the unified virtual assistant, the final output to the user by a virtual assistant head comprised in the unified virtual assistant;
• switch between one or more user contexts in a current user conversation based on roles associated with the one or more user contexts, wherein the one or more user contexts relate to a user context in a previous user conversation, wherein the unified virtual assistant handles and switches across multiple contexts by using agents and tools to work as a multi-index retriever, wherein the unified virtual assistant provides personalized and contextual responses to the user, wherein the unified virtual assistant learns and improves the responses.

One or more non-transitory machine-readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors cause:

• receiving in real time by a virtual assistant engine of a unified virtual assistant, a multi-modal query from a user associated with a role in a user conversation using an enterprise application associated with an enterprise;
• creating by the unified virtual assistant, a user context for the user conversation based on the role associated with the user, wherein the user context is a context which contains information related to a current session of chat being done by a specific user, wherein the user context enables to maintain linking of sequential and back-to-back queries to have a correlation being done in multiple queries in correct manner, wherein the user context is created by instantiating object of chat agent custom class and storing in session provided by UI library, wherein the UI library is used to create a user interface of the unified virtual assistant;
• generating by the unified virtual assistant, an optimized prompt for the multi-modal query corresponding to the user context, based on a set of prompt concepts, wherein the optimized prompt is generated by passing a required prefix or suffix as an input to an agent along with query of the user as another parameter, wherein the agent then modifies the prompt with specified prefix or suffix to get the best response, wherein the passed prefix or suffix are changed or modified dynamically based on the role of the user;
• training a large language model (LLM) using an enterprise context corresponding to the enterprise and a set of user contexts stored in a database;
• generating by the unified virtual assistant, a response corresponding to the optimized prompt from a large language model (LLM) using a customized tool array, wherein the customized tool array comprises a set of tools with each tool further comprising a set of parameters including a tool description, wherein an optimal tool from the customized tool array of the virtual assistant engine is determined based on comparison of the optimal prompt to a set of parameters comprised in the customized tool array, wherein the agent using the LLM finds the optimal tool by executing the tool, halting till tool's response is received as an observation for the LLM and decides the next action based on all preceding responses;
• formatting by the unified virtual assistant, the response to obtain a final output using an output parser;
• and
• providing by the unified virtual assistant, the final output to the user by a virtual assistant head comprised in the unified virtual assistant;
• switching between one or more user contexts in a current user conversation based on roles associated with the one or more user contexts, wherein the one or more user contexts relate to a user context in a previous user conversation, wherein the unified virtual assistant handles and switches across multiple contexts by using agents and tools to work as a multi-index retriever, wherein the unified virtual assistant provides personalized and contextual responses to the user, wherein the unified virtual assistant learns and improves the responses.