# US 9,436,692 B1 · Large scale video analytics architecture > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,436,692 B1 - Document ID: US9436692B1 - Status: Granted - Kind: B1 - Application number: 13/926,130 - Filed: 2013-06-25 - Granted: 2016-09-06 - Earliest claimed: 2013-06-25 - Assignees: EMC Corporation - Applicants: EMC Corporation - Inventors: ChunSheng Fang - Web page: https://patentgrep.com/patent/US9436692B1 ## CPC - G06F 16/71: Indexing; Data structures therefor; Storage structures - G06F 16/116: {Details of conversion of file system types or formats} - G06F 16/182: Distributed file systems - G06F 16/316: {Indexing structures} - G06F 16/783: using metadata automatically derived from the content - G06F 16/951: Indexing; Web crawling techniques - G11B 27/10: Indexing; Addressing; Timing or synchronising; Measuring tape travel - H04N 21/8456: {by decomposing the content in the time domain, e.g. in time segments} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A video file is split into a plurality of chunks. At least a subset of the chunks is processed in parallel, including by detecting one or more moving objects and computing for each detected moving object a visual key and an associated attribute value. The visual key and the associated attribute value are provided as output. ## Claims ### Claim 1 A method, comprising: splitting a video file into a plurality of chunks respectively comprising unstructured video data, wherein the splitting the video file into the plurality of chunks comprises in the event that a boundary corresponding to at least one of the plurality of chunks does not conform to a Group of Pictures boundary associated with a first Group of Pictures, sending a first subset of one or more frames corresponding to the first Group of Pictures to a processor that is to process a second subset of one or more frames corresponding to the first Group of Pictures; processing at least a subset of the chunks in parallel, including by detecting one or more moving objects and computing for each detected moving object a visual key and an associated attribute value, wherein the attribute value comprises one or more of a set of one or more coordinates indicating a geo-location corresponding to a location of the moving object in an image frame with which a key-value pair is associated and a timestamp associated with the image frame with which the key-value pair is associated; providing the visual key and the associated attribute value as output; and storing, in a database, the visual key and the associated attribute value as structured information in a searchable format. ### Claim 2 (depends on 1) The method of claim 1, wherein the video file comprises a standard video file format. ### Claim 3 (depends on 2) The method of claim 2, further comprising detecting the standard video file format. ### Claim 4 (depends on 3) The method of claim 3, further comprising transcoding the video file from the standard video file format to distributed file system-friendly format. ### Claim 5 (depends on 1) The method of claim 1, wherein each chunk is processed at least in part by a map stage to which the chunk is provided as input. ### Claim 6 (depends on 5) The method of claim 5, wherein the visual key and the associated attribute value are computed by the map stage and provided as output in the form of a composite key-value pair. ### Claim 7 (depends on 1) The method of claim 1, wherein the key-value pair is sent to a reduce stage determined based at least in part on the visual key. ### Claim 8 (depends on 7) The method of claim 7, wherein the reduce stage is configured to collect key-value pairs associated with a same moving object and to form and store a trajectory data based on the collected key-value pairs. ### Claim 9 (depends on 1) The method of claim 1, further comprising using the visual key and associated attribute value to compute and store structured information associated with at least a portion of the video file. ### Claim 10 (depends on 9) The method of claim 9, wherein the structured information, optional with a thumbnail image for future reference, is stored in a database. ### Claim 11 (depends on 10) The method of claim 10, further comprising using an index associated with the structured information as stored in the database to retrieve at least a portion of said structured information from the database in response to a database query. ### Claim 12 (depends on 11) The method of claim 11, wherein the query comprises a SQL query. ### Claim 13 (depends on 10) The method of claim 10, wherein the database includes an advanced data analytics capability and further comprising using the advanced data analytics capability to perform advanced data analytics processing with respect to at least a portion of the structured information. ### Claim 14 (depends on 9) The method of claim 9, further comprising providing a visual representation of at least a defined subset of said structured information. ### Claim 15 (depends on 14) The method of claim 14, wherein the structured information comprises a trajectory of the moving object and the visual representation comprises a visual representation of the trajectory. ### Claim 16 (depends on 15) The method of claim 15, wherein the trajectory is represented as a line or one or more other graphical elements arranged in the visual representation to indicate the trajectory of the moving object with respect to a static or other background or reference. ### Claim 17 (depends on 1) The method of claim 1, further comprising decoding, in parallel, one or more image frames from a plurality of localized distributed chunks in a Hadoop Distributed File System (HDFS). ### Claim 18 (depends on 1) The method of claim 1, wherein the processing of at least the subset of chunks and computing for each detected moving object the visual key and the associated attribute value comprises: associating each detected moving foreground object with an image frame, a timestamp associated with the image frame, and a fingerprint representation of the corresponding moving foreground object. ### Claim 19 (depends on 18) The method of claim 18, wherein the fingerprint representation of the corresponding moving foreground object is computed based on an image content analysis. ### Claim 20 (depends on 1) The method of claim 1, wherein the database is searchable using natural-language queries. ### Claim 21 (depends on 1) The method of claim 1, further comprising: receiving a query from a client; querying the database based at least in part on the query; in the event that the database includes data that is responsive to the query, providing the client with a query result, the query result including an image displaying an object corresponding to the query result; and in response to receiving a input from the client, the input corresponding to a selection of the query result, retrieving corresponding underlying raw video data of the object corresponding to the query result. ### Claim 22 A system, comprising: a processor; and a memory coupled with the processor, wherein the memory is configured to provide the processor with instructions which when executed cause the processor to: split a video file into a plurality of chunks respectively comprising unstructured video data, wherein to split the video file into the plurality of chunks comprises in the event that a boundary corresponding to at least one of the plurality of chunks does not conform to a Group of Pictures boundary associated with a first Group of Pictures, sending a first subset of one or more frames corresponding to the first Group of Pictures to a processor that is to process a second subset of one or more frames corresponding to the first Group of Pictures; process at least a subset of the chunks in parallel, including by detecting one or more moving objects and computing for each detected moving object a visual key and an associated attribute value, wherein the attribute value comprises one or more of a set of one or more coordinates indicating a geo-location corresponding to a location of the moving object in an image frame with which a key-value pair is associated and a timestamp associated with the image frame with which the key-value pair is associated; provide the visual key and the associated attribute value as output; and store, in a database, the visual key and the associated attribute value as structured information in a searchable format. ### Claim 23 (depends on 22) The system of claim 22, wherein: the video file comprises a standard video file format; and the memory is further configured to provide the processor with instructions which when executed cause the processor to: detect the standard video file format; and transcode the video file from the standard video file format to distributed file system-friendly format. ### Claim 24 (depends on 22) The system of claim 22, wherein: each chunk is processed at least in part by a map stage to which the chunk is provided as input; and the visual key and the associated attribute value are computed by the map stage and provided as output in the form of a composite key-value pair. ### Claim 25 (depends on 22) The system of claim 22, wherein the memory is further configured to provide the processor with instructions which when executed cause the processor to decode, in parallel, one or more image frames from a plurality of localized distributed chunks in a Hadoop Distributed File System (HDFS). ### Claim 26 A computer program product, the computer program product being embodied in a non-transitory tangible computer readable storage medium and comprising computer instructions for: splitting a video file into a plurality of chunks respectively comprising unstructured video data, wherein the splitting the video file into the plurality of chunks comprises in the event that a boundary corresponding to at least one of the plurality of chunks does not conform to a Group of Pictures boundary associated with a first Group of Pictures, sending a first subset of one or more frames corresponding to the first Group of Pictures to a processor that is to process a second subset of one or more frames corresponding to the first Group of Pictures; processing at least a subset of the chunks in parallel, including by detecting one or more moving objects and computing for each detected moving object a visual key and an associated attribute value, wherein the attribute value comprises one or more of a set of one or more coordinates indicating a geo-location corresponding to a location of the moving object in an image frame with which a key-value pair is associated and a timestamp associated with the image frame with which the key-value pair is associated; providing the visual key and the associated attribute value as output; and store, in a database, the visual key and the associated attribute value as structured information in a searchable format. ### Claim 27 (depends on 26) The computer program product of claim 26, further comprising computer instructions for using the visual key and associated attribute value to compute and store structured information associated with at least a portion of the video file. ### Claim 28 (depends on 27) The computer program product of claim 27, wherein the structured information, optional with a thumbnail image for future reference, is stored in a database. ### Claim 29 (depends on 27) The computer program product of claim 27, further comprising computer instructions for providing a visual representation of at least a defined subset of said structured information, wherein the structured information comprises a trajectory of the moving object and the visual representation comprises a visual representation of the trajectory.