# US 10,121,103 B2 · Scalable deep learning video analytics

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,121,103 B2
- Document ID: US10121103B2
- Status: Granted
- Kind: B2
- Application number: 15/374,571
- Filed: 2016-12-09
- Published: 2018-06-14
- Granted: 2018-11-06
- Earliest claimed: 2016-12-09
- Assignees: Cisco Technologies, Inc.
- Applicants: Cisco Technology, Inc.
- Inventors: Hugo Mike Latapie; Enzo Fenoglio; Joseph T. Friel; Andre Surcouf; Pascal Thubert
- Web page: https://patentgrep.com/patent/US10121103B2

## CPC

- G06V 10/82: using neural networks
- G06F 18/2414: {Smoothing the distance, e.g. radial basis function networks [RBFN]}
- G06N 3/045: Combinations of networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/047: Probabilistic or stochastic networks
- G06N 3/096: Transfer learning
- G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/7715: {Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods}
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06T 2207/10004: Still image; Photographic image
- G06T 2207/20084: Artificial neural networks [ANN]
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

In one embodiment, a method includes receiving training data, the training data including training video data representing video of a location in a quiescent state, training a neural network using the training data to obtain a plurality of metrics, receiving current data, the current data including current video data representing video of the location at a current time period, generating a reconstruction error based on the plurality of metrics and the current video data in the embedded space, and generating, in response to determining that the reconstruction error is greater than a threshold, a notification indicative of the location being in a non-quiescent state.

## Claims

### Claim 1

A method comprising:
receiving training data, the training data including training video data representing video of a location in a quiescent state;
training a neural network using the training data to obtain a plurality of metrics;
receiving current data, the current data including current video data representing video of the location at a current time period;
generating a reconstruction error based on the plurality of metrics and the current video data; and
generating, in response to determining that the reconstruction error is greater than a threshold, a notification indicative of the location being in a non-quiescent state.

### Claim 2 (depends on 1)

The method of claim 1, wherein training the neural network includes initializing the neural network based on a related plurality of metrics obtained by training a neural network using training data from a related location.

### Claim 3 (depends on 1)

The method of claim 1, wherein the training data further includes training sensor data and the current data further includes current sensor data, wherein the reconstruction error is further based on the current sensor data.

### Claim 4 (depends on 3)

The method of claim 3, wherein the current sensor data includes at least one of audio data, pollution data, WiFi data, or social media data.

### Claim 5 (depends on 1)

The method of claim 1, further comprising reducing at least one of a frame rate or resolution of the current video data to generate dimensionality-reduced video data, wherein the reconstruction error is based on the dimensionality-reduced video data.

### Claim 6 (depends on 1)

The method of claim 1, further comprising generating object tracking data based on the current video data, wherein the reconstruction error is based on the object tracking data.

### Claim 7 (depends on 6)

The method of claim 6, wherein the object tracking data includes tracklet data representing a position of a respective one of a plurality of objects at a plurality of time instances.

### Claim 8 (depends on 1)

The method of claim 1, further comprising setting the size of a buffer storing a recent portion of the current data based on the reconstruction error.

### Claim 9 (depends on 1)

The method of claim 1, further comprising setting an amount of dimensionality reduction of the current data based on the reconstruction error.

### Claim 10 (depends on 1)

The method of claim 1, further comprising selecting a computing resource to utilize based on the reconstruction error.

### Claim 11 (depends on 1)

The method of claim 1, further comprising, performing, in response to determining that the reconstruction error is greater than a threshold, anomaly analytics upon the current data.

### Claim 12

A system comprising:
one or more processors; and
a non-transitory memory comprising instructions that when executed cause the one or more processors to perform operations comprising:
receive training data, the training data including training video data representing video of a location in a quiescent state;
train a neural network using the training data to obtain a plurality of metrics;
receive current data, the current data including current video data representing video of the location at a current time period;
generate a reconstruction error based on the plurality of metrics and the current video data; and
generate, in response to determining that the reconstruction error is greater than a threshold, a notification indicative of the location being in a non-quiescent state.

### Claim 13 (depends on 12)

The system of claim 12, wherein training data further includes training sensor data and the current data further includes current sensor data, wherein the reconstruction error is further based on the current sensor data.

### Claim 14 (depends on 12)

The system of claim 12, wherein the operations further comprise at least one of setting the size of a buffer storing a recent portion of the current data based on the reconstruction error, setting an amount of dimensionality reduction of the current data based on the reconstruction error, or selecting a computing resource to utilize based on the reconstruction error.