Image-based abnormal event detection

A system, comprising:

• one or more processors and one or more computer storage media storing instructions that are operable, when executed by the one or more processors, to cause the one or more processors to perform operations comprising:
• obtaining a sequential set of images captured by a camera that includes one or more earlier images and a latter image captured after each of the one or more earlier images;
• generating a first set of observed features for at least some of the one or more earlier images in the sequential set of images;
• generating a second set of observed features for the latter image in the sequential set of images;
• accessing stored data indicating features of objects classified as familiar objects;
• identifying, from the first set of observed features, one or more observed features that satisfy similarity criteria for matching a feature of at least one of the familiar objects;
• filtering the identified one or more observed features from the first set of observed features to generate an unfamiliar object subset of the first set of observed features that excludes the identified one or more observed features;
• generating a set of predicted features for the latter image in the sequential set of images using a prediction model and the unfamiliar object subset of the first set of observed features for the one or more earlier images;
• identifying, from the second set of observed features, one or more observed features that satisfy similarity criteria for matching a feature of at least one of the familiar objects;
• filtering the identified one or more observed features from the second set of observed features to generate an unfamiliar object subset of the second set of the second set of observed features that excludes the identified one or more observed features;
• determining that a difference between the set of predicted features for the latter image in the sequential set of images and the unfamiliar object subset of the second set of observed features for the latter image in the sequential set of images satisfies a threshold difference;
• in response to determining that the difference between the set of predicted features for the latter image in the sequential set of images and the unfamiliar object subset of the second set of observed features for the latter image in the sequential set of images satisfies the threshold difference, determining that the difference between the set of predicted features for the latter image in the sequential set of images and the unfamiliar object subset of the second set of observed features for the latter image in the sequential set of images satisfies one or more abnormal event criteria;
• and
• in response to determining that the difference between the set of predicted features for the latter image in the sequential set of images and the unfamiliar object subset of the second set of observed features for the latter image in the sequential set of images satisfies the one or more abnormal event criteria, classifying the sequential set of images as showing an abnormal event.

A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

• obtaining a sequential set of images captured by a camera that includes one or more earlier images and a latter image captured after each of the one or more earlier images;
• generating a first set of observed features for at least some of the one or more earlier images in the sequential set of images;
• generating a second set of observed features for the latter image in the sequential set of images;
• accessing stored data indicating features of objects classified as familiar objects;
• identifying, from the first set of observed features, one or more observed features that satisfy similarity criteria for matching a feature of at least one of the familiar objects;
• filtering the identified one or more observed features from the first set of observed features to generate an unfamiliar object subset of the first set of observed features that excludes the identified one or more observed features;
• generating a set of predicted features for the latter image in the sequential set of images using a prediction model and the unfamiliar object subset of the first set of observed features for the one or more earlier images;
• identifying, from the second set of observed features, one or more observed features that satisfy similarity criteria for matching a feature of at least one of the familiar objects;
• filtering the identified one or more observed features from the second set of observed features to generate an unfamiliar object subset of the second set of the second set of observed features that excludes the identified one or more observed features;
• determining that a difference between the set of predicted features for the latter image in the sequential set of images and the unfamiliar object subset of the second set of observed features for the latter image in the sequential set of images satisfies a threshold difference;
• in response to determining that the difference between the set of predicted features for the latter image in the sequential set of images and the unfamiliar object subset of the second set of observed features for the latter image in the sequential set of images satisfies the threshold difference, determining that the difference between the set of predicted features for the latter image in the sequential set of images and the unfamiliar object subset of the second set of observed features for the latter image in the sequential set of images satisfies one or more abnormal event criteria;
• and
• in response to determining that the difference between the set of predicted features for the latter image in the sequential set of images and the unfamiliar object subset of the second set of observed features for the latter image in the sequential set of images satisfies the one or more abnormal event criteria, classifying the sequential set of images as showing an abnormal event.