# US 2022/0165304 A1 · Trigger-responsive clip extraction based on remote analysis > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2022/0165304 A1 - Document ID: US20220165304A1 - Status: Published application - Kind: A1 - Application number: 17/103,715 - Filed: 2020-11-24 - Published: 2022-05-26 - Earliest claimed: 2020-11-24 - Assignees: NVIDIA Corporation - Applicants: NVIDIA Corporation - Inventors: Milind Ramesh Naphade; Parthasarathy Sriram; Shuo Wang - Web page: https://patentgrep.com/patent/US20220165304A1 ## CPC - G11B 20/10527: {Audio or video recording; Data buffering arrangements (G11B20/12 - G11B20/18 take precedence)} - G06N 3/04: Architecture, e.g. interconnection topology - G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/063: using electronic means - G06N 3/09: Supervised learning - H04N 5/765: Interface circuits between an apparatus for recording and another apparatus - G06N 3/084: Backpropagation, e.g. using gradient descent - G11B 2020/10712: {buffer capacity, e.g. when the buffer capacity is exhausted, buffered data are overwritten with more recent data, accepting that the old data are lost} - G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Intelligent Video Analytics system may be implemented using a distributed computing architecture with edge and remote devices, where the edge devices analyze the video stream and transmit detection data corresponding to time segments to the remote device. The detection data may identify an object (e.g., vehicle, pedestrian, etc.) in the video stream. The remote device analyzes the detection data received from one or more edge devices and generates extraction triggers that are transmitted to the one or more edge devices. When an edge device receives an extraction trigger, the edge device extracts a clip from the video stream and stores the clip to persistent storage. The remote device may then retrieve the clip. The edge devices may perform simple identification operations while the remote device implements complex algorithms to detect events, benefitting from a larger context than is available to the individual edge devices. ## Claims ### Claim 1 A computer-implemented method, comprising: processing, at an edge device, a data stream acquired from a sensor and continuously buffered to produce detection data corresponding to at least one time segment in the data stream; transmitting the detection data to a remote device; receiving, by the edge device, a trigger generated by the remote device in response to processing of the detection data that identifies an event; in response to the trigger, extracting a clip including at least a portion of the data stream that is continuously buffered by the edge device; and storing the clip to a persistent storage by the edge device. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein the data stream is a video stream and the detection data corresponds to an object that is visible in the data stream. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, wherein the data stream is an audio stream and the detection data corresponds to a sound that is audible in the data stream. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, wherein the sensor comprises at least one of a camera or a microphone. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, wherein the sensor comprises one or more of a light detection and ranging (LIDAR) sensor, infrared (IR) sensor, radio detection and ranging (RADAR) sensor, weather sensor, or speed sensor. ### Claim 6 (canceled) ### Claim 7 (depends on 1) The computer-implemented method of claim 1, further comprising storing annotation data corresponding to the clip and included with the trigger to the persistent storage by the edge device. ### Claim 8 (depends on 1) The computer-implemented method of claim 1, wherein at least one additional edge device is configured to transmit additional detection data to the remote device. ### Claim 9 (canceled) ### Claim 10 (depends on 1) The computer-implemented method of claim 1, wherein the analysis is performed by a neural network. ### Claim 11 (depends on 1) The computer-implemented method of claim 1, wherein the detection data is produced by a neural network. ### Claim 12 (depends on 1) The computer-implemented method of claim 1, wherein the edge device includes a buffer from which the clip is extracted, wherein the buffer continuously stores a portion of the data stream as the data stream is acquired and overwrites an oldest portion of the stored data stream. ### Claim 13 (depends on 1) The computer-implemented method of claim 1, wherein a communication service, connected to the edge device via a network, is configured to relay the detection data from the edge device to the remote device and relay the trigger from the remote device to the edge device. ### Claim 14 (depends on 1) The computer-implemented method of claim 1, wherein the remote device is located within a cloud computing environment. ### Claim 15 (depends on 1) The computer-implemented method of claim 1, wherein the steps of processing, transmitting, receiving, extracting, and storing generate data that are used for training, testing, or certifying a neural network model that is employed in a machine, robot, or autonomous vehicle. ### Claim 16 (depends on 1) The computer-implemented method of claim 1, wherein the steps of processing, transmitting, receiving, extracting, and storing are performed by a virtual machine hosted in a computing device that includes at least one of a processor or a graphics processing unit. ### Claim 17 A system, comprising: an edge device coupled to a persistent storage and configured to: process a data stream acquired from a sensor and continuously buffered to produce detection data corresponding to at least one time segment in the data stream; transmit the detection data to a remote device; receive a trigger generated by the remote device in response to processing of the detection data that identifies an event; in response to the trigger, extract a clip including at least a portion of the data stream that is continuously buffered by the edge device; and store the clip to the persistent storage by the edge device. ### Claim 18 (depends on 17) The system of claim 17, wherein the data stream is a video stream and the detection data corresponds to an object that is visible in the data stream. ### Claim 19 (depends on 17) The system of claim 17, wherein the data stream is an audio stream and the detection data corresponds to a sound that is audible in the data stream. ### Claim 20 A non-transitory computer-readable media storing computer instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: processing a data stream acquired from a sensor and continuously buffered to produce detection data corresponding to at least one time segment in the data stream; transmitting the detection data to a remote device; receiving a trigger generated by the remote device in response to processing of the detection data that identifies an event; in response to the trigger, extracting a clip including at least a portion of the data stream that is continuously buffered by the edge device; and storing the clip to a persistent storage by the edge device. ### Claim 21 (depends on 2) The computer-implemented method of claim 2, wherein the event is associated with movement of the object over time. ### Claim 22 (depends on 12) The computer-implemented method of claim 12, wherein a capacity of the buffer accommodates latency from a time starting when the data stream is acquired and ending after the clip is extracted.