Video frame action detection using gated history

A computerized method comprising:

• receiving a video stream comprising a plurality of video frames;
• grouping the plurality of video frames into a set of present video frames and a set of historical video frames, the set of present video frames comprising a current video frame;
• determining a set of attention weights for the set of historical video frames, the set of attention weights indicating how informative a video frame is for predicting action in the current video frame;
• weighting the set of historical video frames with the set of attention weights to produce a set of weighted historical video frames;
• and
• based on at least the set of weighted historical video frames and the set of present video frames, generating an action prediction for the current video frame.

One or more computer storage devices having computer-executable instructions stored thereon, which, on execution by a computer, cause the computer to perform operations comprising:

• receiving a video stream comprising a plurality of video frames;
• grouping the plurality of video frames into a set of present video frames and a set of historical video frames, the set of present video frames comprising a current video frame;
• determining a set of attention weights for the set of historical video frames, the set of attention weights indicating how informative a video frame is for predicting action in the current video frame;
• weighting the set of historical video frames with the set of attention weights to produce a set of weighted historical video frames;
• and
• based on at least the set of weighted historical video frames and the set of present video frames, generating an action prediction for the current video frame.