US 12,182,199 B2 · Systems and methods for searching for events within video content

Abstract

A video management system (VMS) may search for one or more events in a plurality of video streams captured and stored at a plurality of remote sites. The VMS may generate time-stamped metadata for each video stream captured at the remote site. The time-stamped metadata for each video stream may identify one or more objects and/or events occurring in the corresponding video stream as well as an identifier that uniquely identifies the corresponding video stream. Each of the plurality of remote sites may send the time-stamped metadata to a central hub, wherein the time-stamped metadata may be stored in a data lake, and a user may enter a query into a video query engine, wherein the video query engine may be operatively coupled to the central hub.

A method for searching for one or more events in a plurality of video streams captured and stored at a plurality of remote physical sites, the method comprising:

capturing, from a plurality of cameras, a plurality of video streams, wherein each camera of the plurality of cameras captures a respective video stream of the plurality of video streams, and wherein each camera of the plurality of cameras is located at a respective remote physical site;
for each video stream of the plurality of video streams, storing the video stream at the remote physical site corresponding to the location of the camera that captured the video stream;
processing each of the plurality of video streams at the remote physical site corresponding to the location of the camera that captured the video stream, wherein processing each of the plurality of video streams includes identifying one or more objects in the respective video stream and determining one or more object attributes for each of the identified objects, wherein the one or more object attributes include one or more of:
an object description of the respective object;
a size of the respective object;
a color of the respective object;
a shape of the respective object;
a position of the respective object within a respective video frame of the respective video stream;
one or more associations with one or more other objects identified in the respective video stream based at least in part on a distance between the identified objects in the respective video frame and/or time between the identified objects in the respective video stream;
generating, at each remote physical site of the plurality of remote physical sites, time-stamped metadata for each video stream captured at the remote physical site, the time-stamped metadata for each video stream identifying one or more objects captured by the respective video stream as well as an identifier that uniquely identifies the respective video stream;
wherein, for one or more of the objects captured by the respective video stream, the time-stamped metadata including an object identifier for the respective object and one or more object attributes of the respective object including one or more of:
the object description of the respective object;
the position of the respective object within the respective video frame of the respective video stream;
the size of the respective object;
the color of the respective object;
the shape of the respective object;
one or more associations with one or more other objects identified in the respective video stream based at least in part on a distance between the identified objects in the respective video frame and/or time between the identified objects in the respective video stream;
sending, from each remote physical site of the plurality of remote physical sites, the time-stamped metadata over a network to a central hub located physically remote from the remote physical site;
receiving, by the central hub, a query entered by a user via a user device, and providing the query to a video query engine;
the video query engine applying the query to the time-stamped metadata at the central hub;
returning a search result to the user device, wherein the search result identifies one or more matching objects identified by the time-stamped metadata in the plurality of video streams that match the query;
identifying one or more of the plurality of video streams that include one or more of the matching objects;
downloading at least part of one of the plurality of video streams that include one or more of the matching objects from the respective remote physical site;
and
displaying at least part of the downloaded video stream on a display of the user device.

22 more elements

Systems and methods for searching for events within video content

Life of this patent

Systems and methods for searching for events within video content

Life of this patent

Abstract

Claims20