Systems and methods for identifying activities and/or events in media contents based on object data and scene data

A method for use with a system including a hardware processor and a non-transitory memory storing an executable code including an object data module, a scene data module, an image data module, and a plurality of fusion layers of a neural network, the method comprising:

• receiving, using the hardware processor, a media content having video frames;
• extracting object data, by executing the object data module using the hardware processor, from the video frames of the media content;
• generating, using the hardware processor, an average object data representation of the extracted object data;
• extracting scene data, by executing the scene data module using the hardware processor, from the frames of the video media content;
• generating, using the hardware processor, an average scene data representation of the extracted scene data;
• extracting image data, by executing the image data module using the hardware processor, from the frames of the video media content;
• generating, using the hardware processor, an average image data representation of the extracted image data;
• feeding, using the hardware processor, the average object data representation, the average scene data representation and the average image data representation to the plurality of fusion layers of the neural network;
• and
• classifying, by executing the plurality of fusion layers using the hardware processor, an action in the video frames of the media content using the average object data representation, the average scene data representation and the average image data representation.