US 8,428,345 B2Granted

Meta-classifier system for video analytics

AssigneeHoneywell International Inc.

InventorsYunqian Ma, Scott McCloskey

Life of this patent

Estimating the term…

Loading patent…

Abstract

A system for meta-classification having a training phase mechanism and an operational phase mechanism. The training phase mechanism may have a detection and tracking module, a classifier section connected to the detection and tracking module, a feature synthesis module connected to the classifier section, a labeling module connected to the feature synthesis module and a training data module connected to the labeling module. The operational phase mechanism may have a detection and tracking module, a classifier section connected to the detection and tracking module, a feature synthesis module connected to the classifier section and a meta-classification module connected to the feature synthesis module and the training module. The training phase mechanism may provide parameters and settings to the operational phase mechanism.

Claims14

Dependent claims

A system for meta-classification of surveillance video comprising:

a video acquisition mechanism configured to acquire surveillance video;
a training phase mechanism;
and
an operational phase mechanism;
wherein:
the training phase mechanism comprises:
a detection and tracking module configured to add bounding boxes to interesting objects in a training surveillance video acquired by the video acquisition mechanism;
a workstation with a screen configured to allow a human operator to draw bounding boxes on interesting objects in the training surveillance video;
a classifier section connected to the detection and tracking module, the classifier section including a plurality of classifier channels, each of the plurality of classifier channels configured, for the interesting objects in the training surveillance video, to output a list of features and their respective likelihoods of occurrences, wherein the list of features is common between all of the plurality of classifier channels;
a feature synthesis module connected to the classifier section, the feature synthesis module configured to combine the lists of features from the plurality of classifier channels into a combined list of features;
a labeling module configured to apply labels provided by the human operator to the combined list of features and to determine weights of the likelihoods of occurrences;
and
a training data module connected to the labeling module and receiving the weights;
the operational phase mechanism comprises:
a detection and tracking module configured to add bounding boxes to interesting objects in an operational surveillance video acquired by the video acquisition mechanism;
a classifier section connected to the detection and tracking module, the classifier section including a plurality of classifier channels, each of the plurality of classifier channels configured, for the interesting objects in the operational surveillance video, to output a list of features and their respective likelihoods of occurrences;
a feature synthesis module connected to the classifier section, the feature synthesis module configured to combine the lists of features from the plurality of classifier channels into a combined list of features;
and
a meta-classification module connected to the feature synthesis module and the training data module, the meta-classification module configured to determine labels for the interesting objects in the operational surveillance video using the weights from the training data module and the likelihoods of occurrences of the combined list of features from the feature synthesis module.

16 more elements

A method for meta-classification of surveillance video, comprising:

acquiring one or more training video sequences with a video camera;
selecting out one or more objects of interest in the one or more training video sequences with a detection and tracking module;
applying two or more classifier channels to the one or more objects of interest to get classifier descriptor data having likelihoods of occurrences of activities of the one or more objects of interest, wherein all of the two or more classifier channels provide likelihoods of occurrences of a common set of activities;
combining the classifier descriptor data from the two or more classifier channels with a feature synthesis module;
labeling the classifier descriptor data by a human operator;
determining weights for the likelihoods of occurrences of activities of the one or more objects of interest;
acquiring an operational video sequence with a video camera;
selecting a test object from the operational video sequence with the detection and tracking module;
applying the two or more classifier channels to the test object to obtain classifier descriptor data of activities of the test object, having likelihoods of occurrences of activities of the test object;
and
determining a meta-classification label from the classifier descriptor data of activities of the test object and the weights for the likelihoods of occurrences of activities with a meta-classification module.

8 more elements

A meta-classifier system comprising:

a video module;
a detection and tracking module having an object selector, the detection and tracking module configured to receive one or more video sequences from the video module, and the object selector configured to select an object of interest in the one or more video sequences;
a classifier section connected to the detection and tracking module and configured to provide two or more sets of descriptors of activities of the object of interest, wherein each descriptor of an activity of the object of interest indicates a probability of the activity, further wherein each of the two or more sets of descriptors of activities includes probabilities of a common set of activities;
a synthesis module connected to the classifier section and configured to output a combined set of descriptors comprising the two or more sets of descriptors of activities provided by the classifier section;
a labeling module connected to the synthesis module and configured to apply a label provided by a human to the combined set of descriptors and to determine a weight for each descriptor from the combined set of descriptors and the label;
a training module connected to the labeling module and configured to receive the weight of each descriptor as data;
and
a meta-classification module connected to the synthesis module and the training module, the meta-classification module configured to determine labels for a second object of interest in a second video sequence with a second combined set of descriptors and the data from the training module, wherein the second video sequence is subject to processing by the detection and tracking module, the classifier section, and the synthesis module, such processing including generating the second combined set of descriptors.

5 more elements

Dependencies are inferred from the claim text.

Facts

Application: 12/716,940Filed 3 Mar 2010
Published: US 2011/0216964 A18 Sep 2011
Granted: 23 Apr 2013
Term ends: Not estimatedThe source records don’t support an estimate.

Classification4 codes

G06V 20/41
In video content: higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian…
G06F 16/7837
Using metadata automatically derived from the content: using objects detected or recognised in the video content
G06V
Image and video recognition
G06F
Computing and data processing