# US 10,769,913 B2 · Cloud-based video surveillance management system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,769,913 B2 - Document ID: US10769913B2 - Status: Granted - Kind: B2 - Application number: 13/335,591 - Filed: 2011-12-22 - Published: 2013-06-27 - Granted: 2020-09-08 - Earliest claimed: 2011-12-22 - Assignees: Pelco - Applicants: Lei Wang; Hongwei Zhu; Farzin Aghdasi; Greg Millar - Inventors: Lei Wang; Hongwei Zhu; Farzin Aghdasi; Greg Millar - Web page: https://patentgrep.com/patent/US10769913B2 ## CPC - G08B 13/19671: {Addition of non-video data, i.e. metadata, to video stream} - G08B 13/19643: {wherein the cameras play different roles, e.g. different resolution, different camera type, different control type} - G08B 13/19645: {Multiple cameras, each having view on one of a plurality of scenes, e.g. multiple cameras for multi-room surveillance or for tracking an object by view hand-over} - G08B 13/19656: {Network used to communicate with a camera, e.g. WAN, LAN, Internet} - H04N 7/181: {for receiving images from a plurality of remote sources} - H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 21/4828: {for searching programme descriptors (retrieval of video data G06F16/739)} - H04N 21/8133: {specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video programme} - G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Systems and methods are described herein that provide a three-tier intelligent video surveillance management system. An example of a system described herein includes a gateway configured to obtain video content and metadata relating to the video content from a plurality of network devices, a metadata processing module communicatively coupled to the gateway and configured to filter the metadata according to one or more criteria to obtain a filtered set of metadata, a video processing module communicatively coupled to the gateway and the metadata processing module and configured to isolate video portions, of video the content, associated with respective first portions of the filtered set of metadata, and a cloud services interface communicatively coupled to the gateway, the metadata processing module and the video processing module and configured to provide at least some of the filtered set of metadata or the isolated video portions to a cloud computing service. ## Claims ### Claim 1 A method of managing a network-based video surveillance system through a computing system in a gateway between a plurality of network devices and a cloud computing service for reducing the amount of video data received by the cloud computing service, the method comprising: obtaining video content and metadata relating to the video content from a plurality of network devices including first and second network devices that maintain at least some metadata for overlapping geographic areas; filtering the metadata according to one or more criteria to obtain a filtered set of metadata, wherein filtering the metadata comprises: identifying a metadata element obtained from the first network device that corresponds to the area for which the first network device and the second network device maintain the at least some metadata elements; evaluating quality of the identified metadata element according to the one or more criteria; classifying at least a portion of the metadata element as noise metadata when the evaluated quality it is below a threshold of the one or more criteria; determining whether a metadata element corresponding to the metadata element having at least a portion classified as noise has been obtained from the second network device; identifying the metadata element obtained from the first network device as noise if the metadata element corresponding to the metadata element having at least a portion classified as noise has not been obtained from the second network device; and excluding from the filtered set any metadata element identified as noise to create a noise filtered set of metadata; determining video frames that are each associated with the noise filtered set of metadata; and uploading to a cloud computing service at least one of the noise filtered set of metadata or the video frames exclusively associated with the noise filtered set of metadata. ### Claim 2 (depends on 1) The method of claim 1 wherein the plurality of network devices comprise at least one of cameras or video encoders. ### Claim 3 (depends on 1) The method of claim 1 wherein respective ones of the plurality of network devices are associated with respective ones of a plurality of local networks, and wherein each of the plurality of local networks is associated with distinct geographic locations. ### Claim 4 (depends on 1) The method of claim 1, wherein the filtering further comprises evaluating the quality of the metadata based on at least one of spatial relationships within video content corresponding to the metadata or temporal relationships within the video content corresponding to the metadata. ### Claim 5 (depends on 1) The method of claim 1 further comprising processing the video content to generate one or more supplementary metadata elements. ### Claim 6 (depends on 1) The method of claim 1 further comprising: receiving a query of at least one of the metadata or the video content; and processing the query according to one or more predefined rules. ### Claim 7 (depends on 1) The method of claim 1, further comprising: maintaining in the filtered set a metadata element for an object that becomes partially or fully obscured, in accordance with temporal relationships between observed objects and pre-defined for permanence and feature continuity. ### Claim 8 (depends on 1) The method of claim 1, wherein the excluding excludes from the filtered set of metadata at least one metadata element for at least one object classified as noise in response to the at least one object disappearing from the video content within a threshold amount of time from appearing in the video content. ### Claim 9 (depends on 1) The method of claim 1, wherein filtering the metadata further includes excluding from the filtered set at least one metadata element from the video data from at least one of the at least two neighboring network devices from the plurality of network devices for at least one object determined to be a duplicate of an object in video data from another of the at least two neighboring network devices. ### Claim 10 (depends on 1) The method of claim 1, wherein a map representation of an object for either the first portions of the filtered set of metadata or the video frames associated with the respective first portions of the filtered set of metadata is uploaded to the cloud computing service in place of the video content or metadata. ### Claim 11 (depends on 10) The method of claim 10, further comprising: storing, as a backup at the gateway, at least the video content and metadata associated with the representation to be uploaded. ### Claim 12 (depends on 1) The method of claim 1, wherein the metadata element corresponds to an object or event detected from respective video content. ### Claim 13 A network-based video surveillance management system comprising: a gateway configured to obtain video content and metadata relating to the video content from a plurality of network devices including first and second network devices that maintain at least some metadata for overlapping geographic areas; a metadata processing module communicatively coupled to the gateway and configured to filter the metadata according to one or more criteria to obtain a filtered set of metadata, wherein the metadata processing module configured to filter the metadata is configured to: identify a metadata element obtained from the first network device that corresponds to the area for which the first network device and the second network device maintain the at least some metadata elements; evaluate a quality of the identified metadata according to the one or more criteria; classify at least a portion of the metadata having as noise metadata when the evaluated quality is below a threshold of the one or more criteria; determine whether a metadata element corresponding to the metadata element having at least a portion classified as noise has been obtained from the second network device; identify the metadata element obtained from the first network device as noise if the metadata element corresponding to the metadata element having at least a portion classified as noise has not been obtained from the second network device; and exclude from the filtered set any metadata element identified as noise to create a noise filtered set of metadata; a video processing module communicatively coupled to the gateway and the metadata processing module and configured to determine video frames that are each associated with the noise filtered set of metadata; and a cloud services interface communicatively coupled to the gateway, the metadata processing module and the video processing module, the cloud service interface configured to upload to a cloud computing service at least one of the video content associated with the noise filtered set of metadata or the noise filtered set of metadata. ### Claim 14 (depends on 13) The system of claim 13 wherein the plurality of network devices comprise at least one of cameras or video encoders. ### Claim 15 (depends on 13) The system of claim 13, wherein the metadata processing module is further configured to evaluate the quality of the metadata based on at least one of spatial relationships within video content corresponding to the metadata or temporal relationships within the video content corresponding to the metadata. ### Claim 16 (depends on 13) The system of claim 13 wherein the video processing module is further configured to generate one or more supplementary metadata elements based on the video content. ### Claim 17 (depends on 13) The system of claim 13 further comprising a rule engine communicatively coupled to the gateway, the metadata processing module and the video processing module and configured to receive a query of at least one of the metadata or the video content and to process the query according to one or more predefined rules. ### Claim 18 A computer program product residing on a non-transitory processor-executable computer storage medium, the computer program product comprising processor-executable instructions configured to cause a processor of a computing system in a gateway between a plurality of network devices and a cloud computing service to: obtain video content and metadata relating to the video content from a plurality of network devices including first and second network devices that maintain at least some metadata for overlapping geographic areas; filter the metadata according to one or more criteria to obtain a filtered set of metadata, wherein the processor-executable instructions configured to cause the processor to filter the metadata comprise instruction configured to cause the processor to: identify a metadata element obtained from the first network device that corresponds to the area for which the first network device and the second network device maintain the at least some metadata elements; evaluate a quality of the identified metadata element according to the one or more criteria; classify at least a portion of the metadata as noise when the evaluated quality is below a threshold of the one or more criteria; determine whether a metadata element corresponding to the metadata element having at least a portion classified as noise has been obtained from the second network device; identify the metadata element obtained from the first network device as noise if the metadata element corresponding to the metadata element having at least a portion classified as noise has not been obtained from the second network device; and exclude from the filtered set any metadata element identified as noise to create a noise filtered set of metadata; determine video frames that are each associated with the noise filtered set of metadata; and upload to a cloud computing service the noise filtered set of metadata or the video frames associated with the noise filtered set of metadata. ### Claim 19 (depends on 18) The computer program product of claim 18 wherein the plurality of network devices comprise at least one of cameras or video encoders. ### Claim 20 (depends on 18) The computer program product of claim 18 further comprising processor-executable instructions configured to cause the processor to: receive a query of at least one of the metadata or the video content; and process the query according to one or more predefined rules.