Automatic lie detection method and apparatus for interactive scenarios, device and medium

An automatic lie detection apparatus for interactive scenarios, comprising:

• a sample input module used for inputting a to-be-detected sample;
• a sample segmenting module used for segmenting a video, an audio and a text of the to-be-detected sample according to preset time periods to obtain video information, audio information and text information in multiple time periods;
• a short-term feature extraction module used for extracting short-term video features, short-term audio features and short-term text features in the multiple time periods according to the video information, the audio information and the text information in the multiple time periods;
• a long-term feature extraction module used for aggregating the short-term video features, the short-term audio features and the short-term text features in each dialogue in the to-be-detected sample in chronological order to obtain long-term video features, long-term audio features and long-term text features;
• a multi-modal integration module used for integrating the long-term video features, the long-term audio features and the long-term text features by a multi-modal attention integration strategy to obtain a multi-modal feature of the each dialogue in the to-be-detected sample;
• an interactive modeling module used for inputting the multi-modal feature of the each dialogue and speaker information corresponding to the each dialogue to a graph neural network in chronological order to obtain a multi-modal feature integrated with interactive information, wherein the speaker information is recognized by a pre-trained character information recognition network;
• and
• a lie level prediction module used for predicting a lie level of the each dialogue according to the multi-modal feature integrated with the interactive information.