# US 12,003,744 B2 · Hierarchical surveilance video compression repository > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,003,744 B2 - Document ID: US12003744B2 - Status: Granted - Kind: B2 - Application number: 17/509,940 - Filed: 2021-10-25 - Published: 2023-04-27 - Granted: 2024-06-04 - Earliest claimed: 2021-10-25 - Assignees: Sensormatic Electronics Corporation - Applicants: Sensormatic Electronics, LLC - Inventors: Thibaut De Bock - Web page: https://patentgrep.com/patent/US12003744B2 ## CPC - H04N 19/426: using memory downsizing methods - H04N 21/8153: {comprising still images, e.g. texture, background image} - G06F 16/2246: {Trees, e.g. B+trees} - H04N 7/181: {for receiving images from a plurality of remote sources} - H04N 19/136: Incoming video signal characteristics or properties - H04N 19/176: the region being a block, e.g. a macroblock - H04N 19/23: with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic - H04N 19/503: involving temporal prediction (adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109) - H04N 19/91: Entropy coding, e.g. variable length coding [VLC] or arithmetic coding - H04N 21/234318: {by decomposing into objects, e.g. MPEG-4 objects} - H04N 21/845: Structuring of content, e.g. decomposing content into time segments - G06F 16/783: using metadata automatically derived from the content - H04N 7/18: Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Apparatus and methods for processing video surveillance data includes training a data repository, using a first plurality of surveillance video files including a first plurality of video frames, to identify macroblocks of the video frames representing average content of the first plurality of surveillance video files. An ordered data structure is generated by sorting the plurality of macroblocks of video frames based on image differences within the plurality of macroblocks. The ordered data structure includes a root node. A second plurality of surveillance video files including a second plurality of video frames is received. The second plurality of video frames is inserted into the generated ordered data structure. References to the generated ordered data structure are stored in the data store for each frame of the second plurality of video frames along with a difference between corresponding video frames and references. ## Claims ### Claim 1 A method for processing video surveillance data comprising: calculating average pixel values of corresponding macroblocks in intraframes in each of a first plurality of surveillance video files comprising a first plurality of video frames over a period of time; generating an ordered data structure of a plurality of macroblocks of video frames contained in each of the first plurality of surveillance video files by sorting the plurality of macroblocks of video frames based on image differences within the plurality of macroblocks, wherein the ordered data structure includes a root node, wherein the root node comprises a frame that comprises one or more macroblocks having the average pixel values; receiving a second plurality of surveillance video files comprising a second plurality of video frames; inserting the second plurality of video frames into the ordered data structure; and storing, in a data repository, references to the ordered data structure for each frame of the second plurality of video frames and a difference between corresponding video frames and the references. ### Claim 2 (depends on 1) The method of claim 1, further comprising: retrieving the ordered data structure, in response to receiving a retrieval request from a user; and generating at least a portion of a surveillance video file based on the retrieval request using the ordered data structure. ### Claim 3 (depends on 1) The method of claim 1, further comprising: sending, to the data repository, by one or more image capture devices providing at least one of the first plurality of surveillance video files or the second plurality of surveillance video files, metadata containing at least image capture device information and frame information, prior to calculating the average pixel values. ### Claim 4 (depends on 1) The method of claim 1, further comprising performing entropy encoding to reduce differences between analyzed frames contained in the second plurality of surveillance video files. ### Claim 5 (depends on 1) The method of claim 1, wherein each branch of the ordered data structure represents a macroblock representation of frames having differences not exceeding a predefined threshold. ### Claim 6 (depends on 5) The method of claim 5, wherein each branch of the ordered data structure is associated with a unique identifier stored in the data repository. ### Claim 7 (depends on 1) The method of claim 1, wherein a format of the first plurality of surveillance video files and/or the second plurality of surveillance video files comprises one or more of: H.264, H.265 or H.266. ### Claim 8 A system for processing video surveillance data comprising: a hardware processor configured to: calculate average pixel values of corresponding macroblocks in intraframes in each of a first plurality of surveillance video files comprising a first plurality of video frames over a period of time; generate an ordered data structure of a plurality of macroblocks of video frames contained in each of the first plurality of surveillance video files by sorting the plurality of macroblocks of video frames based on image differences within the plurality of macroblocks, wherein the ordered data structure includes a root node, wherein the root node comprises a frame that comprises one or more macroblocks having the average pixel values; receive a second plurality of surveillance video files comprising a second plurality of video frames; insert the second plurality of video frames into the ordered data structure; and store, in a data repository, references to the ordered data structure for each frame of the second plurality of video frames and a difference between corresponding video frames and references. ### Claim 9 (depends on 8) The system of claim 8, wherein the hardware processor is further configured to: retrieve the ordered data structure, in response to receiving a retrieval request from a user; and generate at least a portion of a surveillance video file based on the retrieval request using the ordered data structure. ### Claim 10 (depends on 8) The system of claim 8, wherein the hardware processor is further configured to: send, to the data repository, by one or more image capture devices providing at least one of the first plurality of surveillance video files or the second plurality of surveillance video files, metadata containing at least image capture device information and frame information, prior to calculating the average pixel values. ### Claim 11 (depends on 8) The system of claim 8, wherein the hardware processor is further configured to perform entropy encoding to reduce differences between analyzed frames contained in the second plurality of surveillance video files. ### Claim 12 (depends on 8) The system of claim 8, wherein each branch of the ordered data structure represents a macroblock representation of frames having differences not exceeding a predefined threshold. ### Claim 13 (depends on 12) The system of claim 12, wherein each branch of the ordered data structure is associated with a unique identifier stored in the data repository. ### Claim 14 (depends on 8) The system of claim 8, wherein a format of the first plurality of surveillance video files and/or the second plurality of surveillance video files comprises one or more of H.264, H.265 or H.266.