Method and system for integrated multimodal input processing for virtual agents

A computer system multimodal input processing for a virtual agent comprising, the computer system comprising:

• one or more computer processors, one or more computer readable memories, one or more computer readable storage devices, and program instructions stored on the one or more computer readable storage devices for execution by the one or more computer processors via the one or more computer readable memories, the program instructions comprising:
• obtaining a multimodal input by the virtual agent from a user, wherein the virtual agent employs an Artificial Intelligence (AI) model;
• identifying a plurality of principal entities within the multimodal input;
• extracting information about each entity of the plurality of principal entities;
• and
• generating a response based on the extracted information.

A non-transitory computer-readable storage medium having stored thereon computer executable instruction which when executed by one or more processors, cause the one or more processors to carry out operations for multimodal input processing for a virtual agent, the operations comprising perform the operations comprising:

• obtaining a multimodal input by the virtual agent from a user, wherein the virtual agent employs an Artificial Intelligence (AI) model;
• identifying a plurality of principal entities within the multimodal input;
• extracting information about each entity of the plurality of principal entities;
• and
• generating a response based on the extracted information.