Methods and systems to predict activity in a sequence of images

A method to determine activity in a sequence of successively acquired images of a scene, comprising:

• acquiring the sequence of images;
• forming, for each image in the sequence of images, a feature block of features extracted from the image;
• determining, for each image in the sequence of images, image specific information, wherein the image specific information includes a weighting indicating Image importance for the image and one or more likelihoods of one or more activities;
• storing the formed feature blocks and the determined image specific information, wherein forming the feature block of features extracted from the image and determining the image specific information comprises retrieving a previously determined feature block and image specific information for the image;
• normalizing, for each image in the sequence of images, the weighting to form a normalized weighting;
• combining, for each image in the sequence of images, the normalized weighting and feature block to form a weighted feature block;
• passing a plurality of weighted feature blocks through a predictive module to determine an activity in the sequence of images;
• and
• outputting a result comprising the determined activity in the sequence of images.

A method for training a predictive module to determine activity in a sequence of successively acquired images of a scene, the method comprising:

• a) acquiring a sequence of images of a scene labelled according to an activity in the scene;
• b) processing the sequence including forming a feature block of features extracted from each image and determining image specific information including a weighting for the image and an activity prediction for the image;
• c) normalizing the determined weightings to form a normalized weighting for each image in the sequence of images;
• d) for each image in the sequence of images, combining the associated normalized weighting and associated feature block to form a weighted feature block;
• e) passing a combination of the weighted feature blocks through a predictive module to determine an activity in the sequence of images;
• f) for each image in the sequence, comparing the activity prediction for the image to the label for the image, and using the result of the comparison to improve the activity prediction for the image;
• g) comparing the determined activity in the sequence of images to the label for the sequence and using the result of the comparison to improve the determination of the activity in the sequence of images;
• and
• h) repeating steps a) to g) for successive sequences of images until a condition is met.

A camera system, comprising:

• sensors;
• memory;
• and
• one or more processors configured to perform operations comprising:
• acquiring a sequence of images;
• forming, for individual images in the sequence of images, a feature block of features extracted from the image;
• determining, for individual images in the sequence of images, image specific information, wherein the image specific information includes a weighting for the image and one or more likelihoods of one or more activities;
• normalizing, for individual images in the sequence of images, the weighting to form a normalized weighting;
• combining, for individual images in the sequence of images, the normalized weighting and feature block to form a weighted feature block;
• passing a plurality of weighted feature blocks through a predictive module to determine an activity in the sequence of images;
• comparing the determined activities with a most likely image activity in the image specific information;
• validating the determined activity based on the comparison;
• and
• outputting a result comprising the validated activity in the sequence of images.