Video action detection method based on convolutional neural network

A video action detection method based on a convolutional neural network (CNN), wherein the convolutional neural network comprises a convolutional layer, a common pooling layer, a temporal-spatial pyramid pooling layer and a full connection layer, wherein the outputs of the convolutional neural network comprise a category classification output layer and a time localization calculation result output layer, the video motion detection method comprising:

• Step 1:
• in a training phase, performing the following steps:
• Step 11) inputting a training video in a CNN model to obtain a feature map;
• Step 12) acquiring segments of different lengths in the training video, and selecting positive samples and negative samples from the actual video action segments (ground truth) as training samples;
• Step 13) inputting the corresponding feature region of the training samples in the feature map into the temporal-spatial pyramid pooling layer to obtain a feature expression of uniform size;
• Step 14) inputting the features of the uniform size into the full connection layer, defining a Loss Function, obtaining a loss value;
• performing backpropagation, adjusting the parameters in the model, and performing training;
• and
• Step 15) gradually reducing the learning rate of training;
• obtaining the trained model when the training loss is no longer falling;
• and
• Step 2:
• in a detection phase, performing the following steps:
• Step 21) inputting an entire video to be detected into the trained model obtained in Step 15);
• Step 22) extracting segments of different lengths in the video to-be-detected, acquiring the feature regions of the corresponding segments in the feature layer of the network, and inputting into the temporal-spatial pyramid pooling layer to obtain a feature expression of uniform size;
• and
• Step 23) discriminating the features of uniform size, and obtaining a classification confidence based on the category classification output layer;
• selecting the classification with the highest confidence, and obtaining the category of the action occurring in the video;
• obtaining a start time and an end time of the action according to time location output from the output layer, thereby fulfilling video action detection.