System and Method for Automated Digital Twin Behavior Modeling for Multimodal Conversations

A computing device comprising:

• a computer-readable medium storing instructions for digital twin behavioral modeling;
• and
• a processor configured to execute the instructions to cause a system including at least a multimodal dialog manager and a digital twin platform to:
• receive one or more multimodal queries or conversations;
• parse the multimodal queries or conversations for content;
• recognize and sense one or more multimodal content from the parsed content;
• train the multimodal dialog manager and a virtual human, wherein to train, the processor is further configured to execute instructions to:
• process social simulations which model behavior of humans and virtual humans, wherein each pair of a human and a virtual human form digital twins;
• and
• transfer knowledge through the social simulations to the virtual humans, wherein social and functional behavior of humans is transferred to the virtual humans via the digital twin platform to generate a learned digital twin behavior model;
• and
• generate responses to the multimodal queries or conversations based on the learned digital twin behavior model.