Weakly supervised video activity detection method and system based on iterative learning

A weakly supervised video activity detection system based on iterative learning, comprising:

• A spatial-temporal feature extraction module, configured to extract spatial-temporal features of a video that contains actions, the spatial-temporal features being divided into the spatial-temporal features in the training set and the spatial-temporal features in the test set;
• a neural network model group construction module, configured to construct a neural network model group, the neural network model group including at least two neural network models, an input of each neural network model being the spatial-temporal feature in the training set, and an output of each neural network model being a class activation sequence, a pseudo temporal label and a video feature of the spatial-temporal feature in the training set in the corresponding neural network model;
• a first training module, configured to train a first neural network model according to the class label of the video, the class activation sequence output by the first neural network model, and the video feature output by the first neural network model, the first neural network model being the first neural network model in the neural network model group;
• an iterative training module, configured to train the next neural network model according to the class label of the video, the pseudo temporal label output by the current neural network model, the class activation sequence output by the next neural network model, and the video feature output by the next neural network model;
• an accuracy detection module, configured to input the spatial-temporal features in the test set into the various neural network models, and respectively perform action detection on each corresponding test video in a test set according to the class activation sequences output by the various neural network models to obtain the detection accuracy of the various neural network models;
• and
• an action detection module, configured to perform action detection on the test video according to the neural network model corresponding to the highest detection accuracy value.