Real-time anomaly detection of crowd behavior using multi-sensor information

A system for detecting an anomaly in crowd behavior, the system comprising

• one or more sensors for observing a crowd in motion;
• storage;
• and
• at least one processor configured to:
• receive sensor data from the one or more sensors, the sensor data representing the crowd in motion;
• partition the sensor data into a set of local areas, each local area forming a neighborhood for analyzing the crowd in motion;
• for each local area in the set of local areas,
• characterize motion in the local area to determine a set of real-time estimates of motion of sub-populations in the local area based at least in part on the sensor data, each sub-population characterized by a pattern of motion based at least in part on sensor data collected over a longer-term time duration describing motion in the sub-population, the longer-term time duration including at least one of minutes, hours, days, weeks, seasons, and years;
• provide a crowd model for each local area, each model representing dynamics of continuous functions describing expected motion near each local area;
• determine a set of parametric values of the crowd model based at least in part on the real-time estimates of the motion of the sub-populations in the local area, to correlate the set of parametric values with a short-time evolution of the motion of the sub-populations in the local area;
• learn and adapt a set of auxiliary stochastic models based at least in part on evolution of the parametric values of the crowd model over time, the set of auxiliary stochastic models characterizing substantially normal evolution of the parametric values of the crowd model over time associated with each local area;
• and
• identify an occurrence of a potential anomaly associated with the local area by comparing predictions from an auxiliary stochastic model in the set of auxiliary stochastic models with the set of parametric values of the crowd model based at least in part on the real-time estimates of the motion of the sub-populations in the local area.

A non-transitory computer program product for detecting an anomaly in crowd behavior, the non-transitory computer program product tangibly embodied in a computer-readable medium, the non-transitory computer program product including instructions operable to cause a data processing apparatus to:

• receive sensor data from one or more sensors, the sensor data representing a crowd in motion;
• partition the sensor data into a set of local areas, each local area forming a neighborhood for analyzing the crowd in motion;
• for each local area in the set of local areas,
• characterize motion in the local area to determine a set of real-time estimates of motion of sub-populations in the local area based at least in part on the sensor data, each sub-population characterized by a pattern of motion based at least in part on sensor data collected over a longer-term time duration describing motion in the sub-population, the longer-term time duration including at least one of minutes, hours, days, weeks, seasons, and years;
• provide a crowd model for each local area, each model representing dynamics of continuous functions describing expected motion near each local area;
• determine a set of parametric values of the crowd model based at least in part on the real-time estimates of the motion of the sub-populations in the local area, to correlate the set of parametric values with a short-time evolution of the motion of the sub-populations in the local area;
• learn and adapt a set of auxiliary stochastic models based at least in part on evolution of the parametric values of the crowd model over time, the set of auxiliary stochastic models characterizing substantially normal evolution of the parametric values of the crowd model over time associated with each local area;
• and
• identify an occurrence of a potential anomaly associated with the local area by comparing predictions from an auxiliary stochastic model in the set of auxiliary stochastic models with the set of parametric values of the crowd model based at least in part on the real-time estimates of the motion of the sub-populations in the local area.