# US 11,030,240 B1 · Systems and methods for efficiently sending video metadata > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,030,240 B1 - Document ID: US11030240B1 - Status: Granted - Kind: B1 - Application number: 16/792,852 - Filed: 2020-02-17 - Granted: 2021-06-08 - Earliest claimed: 2020-02-17 - Assignees: Honeywell International Inc. - Applicants: Honeywell International Inc. - Inventors: Kirupakar Janakiraman; Baskaran Muthusamy; Keyurbhai Patel - Web page: https://patentgrep.com/patent/US11030240B1 ## CPC - G06F 16/71: Indexing; Data structures therefor; Storage structures - H04N 21/235: Processing of additional data, e.g. scrambling of additional data or processing content descriptors - G06F 16/783: using metadata automatically derived from the content - G06F 16/7867: {using information manually generated, e.g. tags, keywords, comments, title and artist information, manually generated time, location and usage information, user ratings} - G06N 20/00: Machine learning - H04N 21/26603: {for automatically generating descriptors from content, e.g. when it is not made available by its provider, using content analysis techniques} - H04N 21/2743: Video hosting of uploaded data from client - H04N 21/84: Generation or processing of descriptive data, e.g. content descriptors {(systems specially adapted for using meta-information in broadcast systems H04H60/73)} - H04N 21/8547: involving timestamps for synchronizing content - G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks - H04N 21/23418: {involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A video management system may send time-stamped metadata corresponding to a video stream across a communication path having a limited bandwidth. Time-stamped metadata for a first reference video frame which identifies objects in the reference frame may be generated, and sent across the communication path. Time-stamped metadata for each of a plurality of first delta video frames following the first reference video frame may be generated, the time-stamped metadata for each of the plurality of first delta video frames may identify changes in detected objects relative to the objects identified in the time-stamped metadata for the first reference video frame. The time-stamped metadata for each of the plurality of first delta video frames may be sent across the communication path. ## Claims ### Claim 1 A method for sending time-stamped metadata corresponding to a video stream across a communication path having a limited bandwidth, the video stream including a plurality of sequential video frames, the method comprising: generating time-stamped metadata for a first reference video frame of the plurality of sequential video frames of the video stream, the time-stamped metadata for the first reference video frame identifying objects detected in the first reference video frame; sending the time-stamped metadata for the first reference video frame across the communication path; generating time-stamped metadata for each of a plurality of first delta video frames following the first reference video frame, the time-stamped metadata for each of the plurality of first delta video frames identifying changes in detected objects relative to the objects identified in the time-stamped metadata for the first reference video frame; sending the time-stamped metadata for each of the plurality of first delta video frames across the communication path; generating time-stamped metadata for a second reference video frame of the plurality of sequential video frames of the video stream, the second reference video frame following the plurality of first delta video frames, the time-stamped metadata for the second reference video frame identifying objects detected in the second reference video frame; sending the time-stamped metadata for the second reference video frame across the communication path; generating time-stamped metadata for each of a plurality of second delta video frames following the second reference video frame, the time-stamped metadata for each of the plurality of second delta video frames identifying changes in detected objects relative to the objects identified in the time-stamped metadata for the second reference video frame; and sending the time-stamped metadata for each of the plurality of second delta video frames across the communication path. ### Claim 2 (depends on 1) The method of claim 1, wherein the time-stamped metadata for each of the first reference video frame and each of the plurality of first delta video frames identifies objects and associations between objects identified in the corresponding video frame. ### Claim 3 (depends on 2) The method of claim 2, wherein the associations between objects includes a distance between objects. ### Claim 4 (depends on 2) The method of claim 2, wherein the associations between objects are represented using a Spatial Temporal Regional Graph (STRG) in the time-stamped metadata. ### Claim 5 (depends on 1) The method of claim 1, wherein for each detected object, the time-stamped metadata identifies an unique object identifier along with one or more of an object description, an object position, and an object size. ### Claim 6 (depends on 5) The method of claim 5, wherein the time-stamped metadata for each of the plurality of first delta video frames identifies changes in detected objects relative to the objects identified in the time-stamped metadata for the first reference video frame by identifying a change in an objects position, size, and/or association with one or more other detected objects. ### Claim 7 (depends on 1) The method of claim 1, wherein for each detected object, the time-stamped metadata identifies an unique object identifier along with one or more of an object description, an object position, an object size, and an association with one or more other detected objects. ### Claim 8 (depends on 7) The method of claim 7, wherein the time-stamped metadata for each of the plurality of first delta video frames identifies a change in detected objects relative to the objects identified in the time-stamped metadata for the first reference video frame by identifying a new object that is not present in the first reference video frame. ### Claim 9 (depends on 1) The method of claim 1, wherein the number of the plurality of first delta video frames following the first reference video frame and the number of the plurality of second delta video frames following the second reference video frame are the same. ### Claim 10 (depends on 1) The method of claim 1, wherein the number of the plurality of first delta video frames following the first reference video frame and the number of the plurality of second delta video frames following the second reference video frame are different. ### Claim 11 (depends on 10) The method of claim 10, wherein the number of the plurality of first delta video frames following the first reference video frame is dependent on an amount of time-stamped metadata generated for each of the plurality of first delta video frames relative to an expected size of the time-stamped metadata if a new reference video frame were taken. ### Claim 12 A video processing system for sending time-stamped metadata corresponding to a video stream across a communication path having a limited bandwidth, the video stream including a plurality of sequential video frames, the video processing system comprising: a memory; one or more processor operatively coupled to the memory, the one or more processors configured to: generate time-stamped metadata for a first reference video frame of the plurality of sequential video frames of the video stream, the time-stamped metadata for the first reference video frame identifying objects detected in the first reference video frame; store the time-stamped metadata for the first reference video frame in the memory; send the time-stamped metadata for the first reference video frame across the communication path; generate time-stamped metadata for each of a plurality of first delta video frames following the first reference video frame, the time-stamped metadata for each of the plurality of first delta video frames identifying changes in detected objects relative to the objects identified in the time-stamped metadata for the first reference video frame; store the time-stamped metadata for each of the plurality of first delta video frames in the memory; send the time-stamped metadata for each of the plurality of first delta video frames across the communication path; generate time-stamped metadata for a second reference video frame of the plurality of sequential video frames of the video stream, the second reference video frame following the plurality of first delta video frames, the time-stamped metadata for the second reference video frame identifying objects detected in the second reference video frame; store the time-stamped metadata for the second reference video frame in the memory; send the time-stamped metadata for the second reference video frame across the communication path; generate time-stamped metadata for each of a plurality of second delta video frames following the second reference video frame, the time-stamped metadata for each of the plurality of second delta video frames identifying changes in detected objects relative to the objects identified in the time-stamped metadata for the second reference video frame; store the time-stamped metadata for each of the plurality of second delta video frames in the memory; and send the time-stamped metadata for each of the plurality of second delta video frames across the communication path. ### Claim 13 (depends on 12) The video processing system of claim 12, wherein the time-stamped metadata for each of the first reference video frame and each of the plurality of first delta video frames identifies objects and associations between objects identified in the corresponding video frame. ### Claim 14 (depends on 13) The video processing system of claim 13, wherein the associations between objects includes a distance between objects. ### Claim 15 (depends on 13) The video processing system of claim 13, wherein the associations between objects are represented using a Spatial Temporal Regional Graph (STRG) in the time-stamped metadata. ### Claim 16 (depends on 12) The video processing system of claim 12, wherein for each detected object, the time-stamped metadata identifies an unique object identifier along with one or more of an object description, an object position, and an object size. ### Claim 17 (depends on 16) The video processing system of claim 16, wherein the time-stamped metadata for each of the plurality of first delta video frames identifies changes in detected objects relative to the objects identified in the time-stamped metadata for the first reference video frame by identifying a change in an objects position, size, and/or association with one or more other detected objects. ### Claim 18 (depends on 12) The video processing system of claim 12, wherein for each detected object, the time-stamped metadata identifies an unique object identifier along with one or more of an object description, an object position, an object size, and an association with one or more other detected objects. ### Claim 19 A method for receiving time-stamped metadata corresponding to a video stream across a communication path having a limited bandwidth, the video stream including a plurality of sequential video frames, the method comprising: receiving time-stamped metadata for a first reference video frame of the plurality of sequential video frames of the video stream, the time-stamped metadata for the first reference video frame identifying objects detected in the first reference video frame; receiving time-stamped metadata for each of a plurality of first delta video frames following the first reference video frame, the time-stamped metadata for each of the plurality of first delta video frames identifying changes in detected objects relative to the objects identified in the time-stamped metadata for the first reference video frame; receiving time-stamped metadata for a second reference video frame of the plurality of sequential video frames of the video stream, the second reference video frame following the plurality of first delta video frames, the time-stamped metadata for the second reference video frame identifying objects detected in the second reference video frame; receiving time-stamped metadata for each of a plurality of second delta video frames following the second reference video frame, the time-stamped metadata for each of the plurality of second delta video frames identifying changes in detected objects relative to the objects identified in the time-stamped metadata for the second reference video frame; and processing the received time-stamped metadata to identify one or more events in the video stream. ### Claim 20 (depends on 19) The method of claim 19, wherein the time-stamped metadata for each of the first reference video frame and each of the plurality of first delta video frames identifies objects and associations between objects identified in the corresponding video frame.