Method, system, and computer-readable medium for obtaining aggregated multi-scale first and second dimensional receptive field information

A system, comprising:

• a memory module configured to store program instructions;
• a processor module configured to execute the program instructions, which cause the processor module to perform at least one step for classifying a video clip comprising:
• aggregating information from a corresponding combination of all of multi-scale first dimensional receptive fields and each of multi-scale second dimensional receptive fields, so that information from multi-scale first and second dimensional receptive fields corresponding to the multi-scale second dimensional receptive fields is obtained;
• wherein aggregated multi-scale first dimensional receptive field information is further obtained during the information from multi-scale first and second dimensional receptive fields corresponding to the multi-scale second dimensional receptive fields is obtained, and the aggregated multi-scale first dimensional receptive field information is obtained using first dimensional importance;
• wherein the multi-scale first dimensional receptive fields being first dimensional is being spatial and the multi-scale second dimensional receptive fields being second dimensional is being temporal, or the multi-scale first dimensional receptive fields being first dimensional is being temporal and the multi-scale second dimensional receptive fields being second dimensional is being spatial;
• and
• wherein a corresponding first convolutional neural network operation set provides each of the multi-scale second dimensional receptive fields and each operation of the first convolutional neural network operation set has a corresponding first dimensional local to local constraint;
• and
• aggregating the information from the multi-scale first and second dimensional receptive fields to obtain aggregated multi-scale first and second dimensional receptive field information, wherein the aggregated multi-scale first and second dimensional receptive field information is obtained using first and second dimensional importance;
• aggregating the aggregated multi-scale first dimensional receptive field information and the aggregated multi-scale first and second dimensional receptive field information to obtain aggregated multi-scale first and second dimensional receptive field information and multi-scale first dimensional receptive field information;
• wherein information in the aggregated multi-scale first and second dimensional receptive field information and multi-scale first dimensional receptive field information, that is not needed for classifying the video clip, is considered as irrelevant or trivial information based on the first dimensional importance and the first and second dimensional importance;
• and
• classifying the video clip based on the aggregated multi-scale first and second dimensional receptive field information and multi-scale first dimensional receptive field information to obtain a classification output.

A non-transitory computer-readable medium having stored program instructions, which when executed by a processor module, cause the processor module to perform steps for classifying a video clip comprising:

• aggregating information from a corresponding combination of all of multi-scale first dimensional receptive fields and each of multi-scale second dimensional receptive fields, so that information from multi-scale first and second dimensional receptive fields corresponding to the multi-scale second dimensional receptive fields is obtained;
• wherein aggregated multi-scale first dimensional receptive field information is further obtained during the information from multi-scale first and second dimensional receptive fields corresponding to the multi-scale second dimensional receptive fields is obtained, and the aggregated multi-scale first dimensional receptive field information is obtained using first dimensional importance;
• wherein the multi-scale first dimensional receptive fields being first dimensional is being spatial and the multi-scale second dimensional receptive fields being second dimensional is being temporal, or the multi-scale first dimensional receptive fields being first dimensional is being temporal and the multi-scale second dimensional receptive fields being second dimensional is being spatial;
• and
• wherein a corresponding first convolutional neural network operation set provides each of the multi-scale second dimensional receptive fields and each operation of the first convolutional neural network operation set has a corresponding first dimensional local to local constraint;
• and
• aggregating the information from the multi-scale first and second dimensional receptive fields to obtain aggregated multi-scale first and second dimensional receptive field information, wherein the aggregated multi-scale first and second dimensional receptive field information is obtained using first and second dimensional importance;
• aggregating the aggregated multi-scale first dimensional receptive field information and the aggregated multi-scale first and second dimensional receptive field information to obtain aggregated multi-scale first and second dimensional receptive field information and multi-scale first dimensional receptive field information;
• wherein information in the aggregated multi-scale first and second dimensional receptive field information and multi-scale first dimensional receptive field information, that is not needed for classifying the video clip, is considered as irrelevant or trivial information based on the first dimensional importance and the first and second dimensional importance;
• and
• classifying the video clip based on the aggregated multi-scale first and second dimensional receptive field information and multi-scale first dimensional receptive field information to obtain a classification output.