Affective haptic regulation method based on multimodal fusion

An affective haptic regulation system based on multimodal fusion, comprising a haptic optimal parameter adjustment module, a haptic generation module, a visual-auditory generation module, a multi-physiological signal acquisition module, a multi-sensory signal acquisition module, and a multimodal fusion emotion recognition module, wherein the haptic optimal parameter adjustment module automatically solves an optimal haptic parameter according to a difference between current emotion and target emotion of a user, and sends the optimal haptic parameter to the haptic generation module and generates a haptic effect;

• wherein the haptic generation module and the visual-auditory generation module cooperate to generate visual-auditory-haptic fusion stimuli to act on the user, so as to regulate and control emotions of the user;
• wherein the multi-physiological signal acquisition module and the multi-sensory signal acquisition module acquire various physiological signals, audio signals and haptic vibration signals of the user in real time, and the signals are inputted to the multimodal fusion emotion recognition module to detect a current affective state of the user, which is then sent back to the haptic optimal parameter adjustment module to form a closed-loop affective haptic regulation system
• wherein one or more processors are configured to execute instructions to perform functions of the haptic optimal parameter adjustment module and the multimodal fusion emotion recognition module, wherein the haptic optimal parameter adjustment module comprises a haptic parameter optimization model and a haptic parameter solving module;
• and
• wherein the one or more processors are further configured to communicate with or control the haptic generation module, the visual-auditory generation module, the multi-physiological signal acquisition module, and the multi-sensory signal acquisition module;
• and
• wherein as a core of the affective haptic regulation system based on multimodal fusion, the haptic optimal parameter adjustment module automatically seeks for a haptic parameter with the help of an optimization theory according to the difference between the current affective state and the target emotion of the user, and sends the haptic parameter to the haptic generation module to ensure the effectiveness of affective regulation;
• and
• the haptic optimal parameter adjustment module comprises a haptic parameter optimization model and a haptic parameter solving module.