# US 11,907,291 B2 · System for integral analysis and management of video data > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,907,291 B2 - Document ID: US11907291B2 - Status: Granted - Kind: B2 - Application number: 17/923,247 - Filed: 2020-07-02 - Published: 2023-07-20 - Granted: 2024-02-20 - Earliest claimed: 2020-05-07 - Assignees: XII Lab - Applicants: XII Lab - Inventors: Woo Yung Lee; Dae Su Chung - Web page: https://patentgrep.com/patent/US11907291B2 ## CPC - G06F 16/783: using metadata automatically derived from the content - G06F 16/7837: {using objects detected or recognised in the video content} - G06F 16/71: Indexing; Data structures therefor; Storage structures - G06F 16/732: Query formulation - G06F 16/75: Clustering; Classification - G06F 18/23: Clustering techniques - G06F 18/285: {Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system} - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/08: Learning methods - G06N 3/09: Supervised learning - G06V 10/40: Extraction of image or video features - H04N 7/18: Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast - H04N 21/23418: {involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - G06V 2201/10: Recognition assisted with metadata - H04N 21/231: Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion - H04N 21/4828: {for searching programme descriptors (retrieval of video data G06F16/739)} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A system for integrated analysis and management of video data includes a routing server configured to receive video data from an external input video providing device, a video database configured to store the received video data, a video analyzing server cluster including a plurality of video analyzing servers configured to analyze the video data, a metadata database configured to store metadata of the video data, and a video search server configured to search the metadata database and the video database. As described in various embodiments of the present disclosure, the system can perform integrated analysis of various video data received from a number of channels and provide an analysis result, and can also rapidly provide an accurate search result in response to a user's request for search. ## Claims ### Claim 1 A system that comprises a processor coupled to a memory for integrated analysis and management of video data, comprising: a video analyzing server cluster including a plurality of video analyzing servers configured to analyze the video data; a routing server configured to receive video data from an external input video providing device, wherein the routing server is further configured to allocate analysis of the video data to a specific video analyzing server among the plurality of video analyzing servers of the video analyzing server cluster according to characteristics of the received video data; a video database configured to store the received video data; a metadata database configured to store metadata of the video data; and a video search server configured to search the metadata database and the video database. ### Claim 2 (depends on 1) The system for integrated analysis and management of video data of claim 1, wherein the characteristics of the received video data include at least one of a type of a channel that records the video data, a characteristic of the channel, and a type of an object to be recognized among objects recorded in the video data. ### Claim 3 (depends on 1) The system for integrated analysis and management of video data of claim 1, wherein the video database is a NoSQL database based on Hbase. ### Claim 4 (depends on 1) The system for integrated analysis and management of video data of claim 1, wherein the specific video analyzing server allocated the analysis of the video data by the routing server among the plurality of video analyzing servers of the video analyzing server cluster is further configured to analyze the video data to recognize one or more objects included therein and generate the metadata according to a recognition result. ### Claim 5 (depends on 4) The system for integrated analysis and management of video data of claim 4, wherein the specific video analyzing server is further configured to generate inverse features based on the generated metadata, and the metadata database is further configured to store the generated inverse features. ### Claim 6 (depends on 1) The system for integrated analysis and management of video data of claim 1, wherein the video search server is further configured to receive a video search query from an external user device and search at least one of the metadata database and the video database based on the received search query. ### Claim 7 (depends on 6) The system for integrated analysis and management of video data of claim 6, wherein the video search server is further configured to obtain information about video data required in the search query by searching the metadata database based on the search query, and obtain the video data required in the search query by searching the video database based on the obtained information. ### Claim 8 (depends on 1) The system for integrated analysis and management of video data of claim 1, further comprising: a video converter configured to generate a graphic user interface in a predetermined format based on an analysis result of the video analyzing server cluster. ### Claim 9 (depends on 1) The system for integrated analysis and management of video data of claim 1, further comprising: an I/O server configured to transmit an analysis result image of the video analyzing server cluster to an external video output/control device and receive a video search query from an external user device. ### Claim 10 (depends on 1) The system for integrated analysis and management of video data of claim 1, further comprising: a deep learning server configured to learn video data stored in the video database and generate a video analysis model; and a learning data result value database configured to store the video analysis model generated in the deep learning server. ### Claim 11 (depends on 10) The system for integrated analysis and management of video data of claim 10, wherein the deep learning server is further configured to provide at least one of easy training and hard training for video data learning to a user connected through a deep learning platform web interface for a number of users outside the system. ### Claim 12 A method for integrated analysis and management of video data that is performed by one or more computing devices in a video analysis and management system, comprising: a process of receiving video data from any one channel of an input video providing device including a plurality of channels; a process of allocating the received video data to a specific video analyzing server of a video analyzing server cluster including a plurality of video analyzing servers in the video analysis and management system according to characteristics of the received video data; a process of analyzing the video data by the specific video analyzing server to recognize one or more objects included therein; and a process of generating a graphic user interface in a predetermined format based on an analysis result and an object recognition result. ### Claim 13 (depends on 12) The method for integrated analysis and management of video data of claim 12, further comprising: a process of storing the received video data in a video database in the video analysis and management system, wherein the video database is a NoSQL database based on Hbase. ### Claim 14 (depends on 13) The method for integrated analysis and management of video data of claim 13, further comprising: a process of generating metadata of the recognized one or more objects by the specific video analyzing server; a process of generating inverse features based on the generated metadata by the specific video analyzing server; and a process of storing the generated metadata and inverse features in a metadata database in the video analysis and management system. ### Claim 15 (depends on 14) The method for integrated analysis and management of video data of claim 14, further comprising: a process of receiving a search query for a specific object from an external user device; a process of searching at least one of the metadata database and the video database based on the received search query; and a process of transmitting a search result to the external user device. ### Claim 16 (depends on 15) The method for integrated analysis and management of video data of claim 15, further comprising: a process of determining whether the search result is suitable for the search query; and a process of searching at least one of the metadata database and the video database based on the received search query when it is determined not suitable for the search query. ### Claim 17 (depends on 12) The method for integrated analysis and management of video data of claim 12, wherein the process of analyzing the video data by the specific video analyzing server to recognize one or more objects included therein includes: a process of analyzing the video data to recognize one or more objects included therein based on at least one of a video analysis model generated through deep learning based on channel metadata of a channel that provides the video data and a separate video analysis model received from outside of the video analysis and management system. ### Claim 18 A non-transitory computer-readable storage medium that stores a computer program for integrated analysis and management of video data, wherein the computer program includes one or more instructions to be executed by one or more computing devices in a video analysis and management system, and the one or more instructions include: at least one instruction to receive video data from any one channel of an input video providing device including a plurality of channels; at least one instruction to allocate the received video data to a specific video analyzing server of a video analyzing server cluster including a plurality of video analyzing servers in the video analysis and management system according to characteristics of the received video data; at least one instruction to analyze the video data by the specific video analyzing server to recognize one or more objects included therein; and at least one instruction to generate a graphic user interface in a predetermined format based on an analysis result and an object recognition result. ### Claim 19 (depends on 18) The non-transitory computer-readable storage medium of claim 18, wherein the one or more instructions include: at least one instruction to receive a search query for a specific object from an external user device; at least one instruction to search at least one of a metadata database and a video database based on the received search query; and at least one instruction to transmit a search result to the external user device.