Method for complex target identification from mass videos based on human-machine collaboration

A method for complex target identification from monitoring videos based on human-machine collaboration, comprising the following steps:

• step 1:
• establishing and training a target detection model and a reidentification model;
• step 2:
• preprocessing the monitoring videos;
• extracting video frame images from retrieved monitoring video data;
• performing detection analysis on the video frame images using the target detection model, and framing and clipping a plurality of targets in the video frame images to obtain cropped images;
• and extracting a feature of each target of the target plurality of targets in the cropped images using the reidentification model;
• step 3:
• establishing and calibrating electroencephalogram (EEG) classification model;
• and determining, by the EEG classification model, whether a user from which an EEG signal is collected observes a target of interest;
• and
• step 4:
• performing target retrieval from the monitoring videos based on human-machine collaboration, specifically comprising:
• step 4-1, prescreening, by using the reidentification model, the cropped images identified by the target detection model according to a coarse-grained feature of the target of interest;
• selecting a cropped images according to a confidence level by the reidentification model;
• and extracting the video frame images corresponding to the n selected cropped images;
• step 4-2, providing the video frame images obtained after the prescreening in step 4-1 for viewing by the user;
• recording an EEG signal and eye movement information while the user views the video frame images;
• preprocessing the obtained EEG signal and then inputting the preprocessed EEG signal to the EEG classification model;
• when the EEG classification model determines that the user observes the target of interest, processing eye movement data of the user, extracting regions of interest for the user from the video frame images;
• and extracting a candidate target set from the regions of interest;
• step 4-3, selecting a plurality of target images finally needing to be retrieved from the candidate target set obtained in step 4-2;
• and
• step 4-4, extracting a feature from each target image obtained in step 4-3 using the reidentification model, and performing similarity matching on the feature extracted from each target image and the feature of each target of the plurality of targets in the cropped images;
• and taking each video frame image corresponding to the feature having a similarity exceeding a threshold as an image in which the target of interest is present.