Method and apparatus for detecting temporal action of video, electronic device and storage medium

An electronic device, comprising:

• at least one processor;
• and
• a memory communicatively connected with the at least one processor;
• wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform a method for detecting a temporal action of a video, wherein the method comprises:
• successively extracting a plurality of images from the video to be detected in an extraction order from front to back at a preset sampling period, so as to form an image sequence, and extracting a feature of each image in the image sequence, and splicing the features into the initial temporal feature sequence in the order in the image sequence;
• reducing, by a pre-trained video-temporal-action detecting module, the temporal length of the initial temporal feature sequence according to a plurality of preset temporal length reduction ways, so as to obtain a plurality of adjusted reference temporal feature sequences, configuring a plurality of temporal anchor boxes with a preset number and different scales for temporal positions of each reference temporal feature sequence, wherein the preset temporal length reduction ways comprises four reference temporal feature sequences with different temporal lengths T/4, T/8, T/16 and T/32 if the initial temporal feature sequence has a length T, configuring the implicit feature of each temporal anchor box as the feature of the corresponding temporal position in the corresponding reference temporal feature sequence, and taking the reference temporal feature sequence with the longest length in the plural reference temporal feature sequences as a sample, and acquiring the explicit feature of each temporal anchor box by multiplying the sample by a sampling feature constructed for each temporal anchor box in the form of a vector;
• and
• splicing the explicit feature and the implicit feature of each temporal anchor box to obtain a comprehensive feature of each temporal anchor box;
• detecting each temporal anchor box based on the comprehensive feature thereof, and screening a target temporal anchor box containing the specified action, the category of the specified action and the probability that the specified action belongs to the category;
• and
• detecting boundaries of the target temporal anchor box containing the specified action in the initial temporal feature sequence, and acquiring the starting position and the ending position of the video clip corresponding to the target temporal anchor box.

A non-transitory computer readable storage medium with computer instructions stored thereon, wherein the computer instructions are used for causing a computer to perform a method for detecting a temporal action of a video, comprising:

• successively extracting a plurality of images from the video to be detected in an extraction order from front to back at a preset sampling period, so as to form an image sequence, and extracting a feature of each image in the image sequence, and splicing the features into the initial temporal feature sequence in the order in the image sequence;
• reducing, by a pre-trained video-temporal-action detecting module, the temporal length of the initial temporal feature sequence according to a plurality of preset temporal length reduction ways, so as to obtain a plurality of adjusted reference temporal feature sequences, configuring a plurality of temporal anchor boxes with a preset number and different scales for temporal positions of each reference temporal feature sequence, wherein the preset temporal length reduction ways comprises four reference temporal feature sequences with different temporal lengths T/4, T/8, T/16 and T/32 if the initial temporal feature sequence has a length T, configuring the implicit feature of each temporal anchor box as the feature of the corresponding temporal position in the corresponding reference temporal feature sequence, and taking the reference temporal feature sequence with the longest length in the plural reference temporal feature sequences as a sample, and acquiring the explicit feature of each temporal anchor box by multiplying the sample by a sampling feature constructed for each temporal anchor box in the form of a vector;
• and
• splicing, by the video-temporal-action detecting module, the explicit feature and the implicit feature of each temporal anchor box to obtain a comprehensive feature of each temporal anchor box;
• detecting each temporal anchor box based on the comprehensive feature thereof, and screening a target temporal anchor box containing the specified action, the category of the specified action and the probability that the specified action belongs to the category;
• and
• detecting boundaries of the target temporal anchor box containing the specified action in the initial temporal feature sequence, and acquiring the starting position and the ending position of the video clip corresponding to the target temporal anchor box.