Two-stage online detection of action start in untrimmed videos

A system for detection of an action start in a video, the system comprising:

• a memory storing a plurality of processor-executable instructions;
• and
• a processor executing the plurality of processor-executable instructions to perform operations comprising:
• generating, via a classification module, a set of action scores corresponding to a first video frame from the video, based on the first video frame and video frames before the first video frames in the video, each action score indicating a respective probability that the first video frame contains a respective action class;
• receiving, at a localization module that is separate from the classification module, the set of action scores from the classification module;
• generating, via the localization module, an action-agnostic start probability that the first video frame contains an action start based at least in part on the set of action scores;
• and
• generating, via a fusion component, based on the set of action scores and the action-agnostic start probability, a set of action-specific start probabilities, each action-specific start probability corresponding to a start of an action belonging to the respective action class.