Prior-driven supervision for weakly-supervised temporal action localization

A system comprising:

• a computing processor;
• a memory;
• a weakly-supervised temporal action localization (WTAL) model stored in the memory and executable by the processor to
• ingest into a first classifier within the WTAL model a video clip that includes actions of interest for identification;
• identify action characteristics within frames of the video clip, the action characteristics corresponding to known action classes;
• determine an actionness score for each of the frames based upon the action characteristics identified within each of the frames;
• determine a class activation sequence score (CAS) for sequences of the frames based upon a presence or an absence of the action characteristics identified within each of the frames;
• and
• produce one or more base confidence predictions of temporal locations of actions of interest within the video clip by correlating each of the actionness scores with corresponding class activation scores for each of the frames in the sequences frames;
• a snippet generator stored in the memory and executable by the processor to
• generate a training class of action snippets corresponding to known ground-truth actions within the video clip based on the one or more base confidence predictions of temporal locations of actions of interest;
• and
• a prior-driven, weakly-supervised temporal action localization model stored in the memory and executable by the processor to train a second classifier with the training class to identify the actions of interest at a video level.

A method performed by a computing system comprising

• ingesting a data file having training data therein characterized by both sequential and temporal characteristics, the data file including attributes of interest for identification;
• identifying attribute characteristics within data blocks of the training data, the attribute characteristics corresponding to known attribute classes;
• determining an attribute-based score for each of the data blocks based upon the attribute characteristics identified within each of the data blocks;
• determining a class activation sequence score for each of the data blocks based upon a presence or an absence of the attribute characteristics within each of the data blocks;
• correlating each of the attribute-based scores with corresponding class activation scores for each of the data blocks to produce one or more base confidence predictions of temporal locations of attributes of interest within the data file;
• generating a training class of ground-truth attributes based on the one or more base confidence predictions of temporal locations of attributes of interest within one or more sequences of data blocks within the data file;
• training a classifier with the training class to identify attributes of interest at a file level;
• and
• identifying different sequences of data blocks within a different data file that exhibit the attributes of interest by processing the different data file with the classifier at the file level.