Reasoning from surveillance video via computer vision-based multi-object tracking and spatiotemporal proximity graphs

A computer program product comprising

• a non-transitory computer-readable storage medium;
• and
• instructions stored on the non-transitory computer-readable storage medium that, when executed by a processor, causes the processor to:
• receive overhead video footage;
• receive a scene associated with the overhead video footage including a plurality of tracks, each track of the plurality of tracks corresponding to an entity and each track of the plurality of tracks being associated with a bounding box;
• build a spatiotemporal proximity graph based on the plurality of tracks;
• identify, based on the spatiotemporal proximity graph and a distance between a plurality of the entities, a spatiotemporal relationship between the plurality of the entities;
• and
• identify one or more portions of the overhead video footage that correspond to the spatiotemporal proximity graph.