# US 10,373,458 B2 · Automatic threat detection based on video frame delta information in compressed video streams

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,373,458 B2
- Document ID: US10373458B2
- Status: Granted
- Kind: B2
- Application number: 15/492,011
- Filed: 2017-04-20
- Published: 2018-10-25
- Granted: 2019-08-06
- Earliest claimed: 2017-04-20
- Applicants: Deep Sentinel Corp.
- Inventors: David Lee Selinger; Ching-Wa Yip; Chaoying Chen
- Web page: https://patentgrep.com/patent/US10373458B2

## CPC

- G08B 13/19602: {Image analysis to detect motion of the intruder, e.g. by frame subtraction}
- B60R 25/1012: {Zone surveillance means, e.g. parking lots, truck depots}
- G08B 13/196: using television cameras
- G08B 13/19608: {Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position}
- G08B 13/19663: {Surveillance related processing done local to the camera}
- G08B 13/19667: {Details realated to data compression, encryption or encoding, e.g. resolution modes for reducing data volume to lower transmission bandwidth or memory requirements}
- H04N 7/18: Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N 7/181: {for receiving images from a plurality of remote sources}
- H04N 21/21805: {enabling multiple viewpoints, e.g. using a plurality of cameras}
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Apparatus and associated methods relate to identifying objects of interest and detecting motion to automatically detect a security threat as a function of video frame delta information received from a video encoder. In an illustrative example, the video encoder may be an H.264 encoder onboard a video camera. A cloud server may receive the video frame delta information in a compressed video stream from the camera. Threats may be detected by the cloud server processing the video frame delta information in the compressed video stream, without decompression, to identify objects and detect motion. The cloud server may employ artificial intelligence techniques to enhance threat detection by the cloud server. Various examples may advantageously provide increased capacity of a computer tasked with detecting security breaches, due to the significant reduction in the amount of data to be processed, relative to threat detection based on processing uncompressed video streams.

## Claims

### Claim 1

A system to detect an event in compressed video, comprising:
a plurality of surveillance cameras, each of the plurality of surveillance cameras comprising:
a video encoder, configured to compress video and emit frames of a compressed video stream comprising video frame delta information, wherein said video frame delta information excludes redundant data in the form of pixels in each frame that do not change relative to a reference frame and identifies pixels that have changed relative to a prior frame;
a processor, communicatively coupled to the video encoder;
a memory that is not a transitory propagating signal, the memory connected to the processor and encoding computer readable instructions, including processor executable program instructions, the computer readable instructions accessible to the processor, wherein the processor executable program instructions, when executed by the processor, cause the processor to perform operations comprising:
receiving, from the video encoder, a series of video frames comprising video frame delta information; and
identifying objects of interest and detecting motion to detect a security threat as a function of the video frame delta information; and
a cloud server communicatively coupled to the plurality of surveillance cameras, the cloud server comprising:
a second processor, operably coupled to a memory that is not a transitory propagating signal, the memory connected to the second processor and comprising second computer readable instructions, including second processor executable program instructions, the second computer readable instructions accessible to the second processor, wherein the second processor executable program instructions, when executed by the second processor, cause the processor to perform operations comprising:
sending, to at least one surveillance camera, an electronic message comprising a pattern representative of an event to be detected; and
receiving, from the at least one surveillance camera, an electronic message comprising an indication of a detected event based at least in part on said pattern representative of said event,
wherein the cloud server memory further comprises a predictive model adapted to estimate an event probability determined as a function of video frame delta information; the operations performed by the cloud server processor further comprise sending the predictive model to the surveillance camera; and the operations performed by the surveillance camera processor further comprise: receiving the predictive model from the cloud server; estimating an event probability determined as a function of video frame delta information applied as input data to the predictive model; and upon a determination the estimate of the event probability satisfies a predetermined criterion, sending the estimate of the event probability to the cloud server.

### Claim 2 (depends on 1)

The system of claim 1, in which the operations performed by the surveillance camera processor to identify objects of interest and detect motion as a function of the video frame delta information further comprise a technique selected from the group consisting of optical flow, edge detection, boundary detection, feature extraction, speeded up robust features, scale-invariant feature transform, and template matching.

### Claim 3

A system to detect an event in compressed video, comprising:
a plurality of surveillance cameras, each of the plurality of surveillance cameras comprising:
a video encoder, configured to compress video and emit frames of a compressed video stream comprising video frame delta information;
a processor, communicatively coupled to the video encoder;
a memory that is not a transitory propagating signal, the memory connected to the processor and encoding computer readable instructions, including processor executable program instructions, the computer readable instructions accessible to the processor, wherein the processor executable program instructions, when executed by the processor, cause the processor to perform operations comprising:
receiving, from the video encoder, a series of video frames comprising video frame delta information; and
identifying objects of interest and detecting motion to detect a security threat as a function of the video frame delta information; and
a cloud server communicatively coupled to the plurality of surveillance cameras, the cloud server comprising:
a second processor, operably coupled to a memory that is not a transitory propagating signal, the memory connected to the second processor and comprising second computer readable instructions, including second processor executable program instructions, the second computer readable instructions accessible to the second processor, wherein the second processor executable program instructions, when executed by the second processor, cause the processor to perform operations comprising:
sending, to at least one surveillance camera, an electronic message comprising a pattern representative of an event to be detected; and
receiving, from the at least one surveillance camera, an electronic message comprising an indication of a detected event based at least in part on said pattern representative of said event,
wherein the cloud server memory comprises a predictive model adapted to estimate an event probability determined as a function of video frame delta information; the operations performed by the cloud server processor further comprise sending the predictive model to the surveillance camera; and the operations performed by the surveillance camera processor further comprise: receiving the predictive model from the cloud server; estimating an event probability determined as a function of video frame delta information applied as input data to the predictive model; and upon a determination the estimate of the event probability satisfies a predetermined criterion, sending the estimate of the event probability to the cloud server.