Anomalous activity detection for mobile surveillance robots

A system for anomalous activity detection, the system comprising:

• a memory storing instructions;
• one or more Input/Output (I/O) interfaces;
• and
• one or more hardware processors coupled to the memory via the one or more I/O interfaces, wherein the one or more hardware processors are configured by the instructions to:
• select a set of frames from among a plurality of frames continually captured by an ego view camera, wherein Field of View of the ego view camera changes dynamically during navigation, and wherein the set of frames lie within a single pre-defined time step;
• estimate a motion present across a plurality of pixels of each of the set of frames by processing the set of frames based on a hybrid approach comprising (i) fuzzy sets, (ii) random consensus and scene change detection, and (iii) an optical flow computed across the set of frames;
• obtain a net change in each frame at pixel level to segregate static pixels and a Region of Interest (RoI) in each of the sequence of frames by computing Structural Similarity Index Measure (SSIM) between the initial frame and each of the set of frames, wherein the RoI represents pixels associated with the motion of a moving object;
• mask the static pixels in each of the sequence of frames to generate a masked sequence of frames comprising the RoI;
• and
• process (i) the RoI of each of the masked sequence of frames, and (ii) associated estimated motion for the set of frames to predict presence of one of (i) an anomalous activity and (ii) a normal activity across the sequence of frames, wherein the processing comprising:
• feeding (i) the RoI of each of the masked sequence of frames through a first sequence of Convolutional Long short-term memory (ConvLSTM) layers to obtain a first initial feature map of the RoI, and (ii) the associated estimated motion for the set of frames through a second sequence of ConvLSTM layers to obtain a second initial feature map of the estimated motion, for memorizing changes present in the motion of the moving object;
• feeding (i) the first initial feature map to a first convolution three dimensional network, and (ii) the second initial feature map to a second convolution 3D network as input, wherein each of the first convolution 3D network and the second convolution 3D network is a parallel pathway structure resembling Konio-Parvocellular-Magno (KPM) cells of a human brain that utilizes varying strides for processing an input with varying frame rate for generating an intermediate level feature map, wherein the varying frame rates and strides are learnt using grid search, while maintaining a hierarchy as a ratio of the lower stride/frame rate number to higher stride/frame rate;
• and
• concatenating the intermediate level feature map generated by each of the first convolution 3D network and the second convolution 3D network and generating a final feature map by concatenating the intermediate level feature map with a fully connected layer to predict an output as one of the anomalous activity and the normal activity associated with ROI in the set of frames.

One or more non-transitory machine-readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors cause:

• selecting, a set of frames from among a plurality of frames continually captured by an ego view camera, wherein Field of View of the ego view camera changes dynamically during navigation, and wherein the set of frames lie within a single pre-defined time step;
• estimating, via the one or more hardware processors, a motion present across a plurality of pixels of each of the set of frames by processing the set of frames based on a hybrid approach comprising (i) fuzzy sets, (ii) random consensus and scene change detection, and (iii) an optical flow computed across the set of frames;
• obtaining, via the one or more hardware processors, a net change in each frame at pixel level to segregate static pixels and a Region of Interest (RoI) in each of the sequence of frames by computing Structural Similarity Index Measure (SSIM) between the initial frame and each of the set of frames, wherein the RoI represents pixels associated with the motion of a moving object;
• masking, via the one or more hardware processors, the static pixels in each of the sequence of frames to generate a masked sequence of frames comprising the RoI;
• and
• processing, via a trained spatio-temporal neural network executed by the one or more hardware processors (i) the RoI of each of the masked sequence of frames, and (ii) associated estimated motion for the set of frames to predict presence of one of (i) an anomalous activity and (ii) a normal activity across the sequence of frames, wherein the processing comprising:
• feeding (i) the RoI of each of the masked sequence of frames through a first sequence of Convolutional Long short-term memory (ConvLSTM) layers to obtain a first initial feature map of the RoI, and (ii) the associated estimated motion for the set of frames through a second sequence of ConvLSTM layers to obtain a second initial feature map of the estimated motion, for memorizing changes present in the motion of the moving object;
• feeding (i) the first initial feature map to a first convolution three dimensional network, and (ii) the second initial feature map to a second convolution 3D network as input, wherein each of the first convolution 3D network and the second convolution 3D network is a parallel pathway structure resembling Konio-Parvocellular-Magno (KPM) cells of a human brain that utilizes varying strides for processing an input with varying frame rate for generating an intermediate level feature map, wherein the varying frame rates and strides are learnt using grid search, while maintaining a hierarchy as a ratio of the lower stride/frame rate number to higher stride/frame rate;
• and
• concatenating the intermediate level feature map generated by each of the first convolution 3D network and the second convolution 3D network and generating a final feature map by concatenating the intermediate level feature map with a fully connected layer to predict an output as one of the anomalous activity and the normal activity associated with ROI in the set of frames.