Online learning method and system for action recognition

An apparatus comprising:

• a processing device;
• and
• a memory device coupled to the processing device, the memory device having instructions stored thereon that, in response to execution by the processing device, cause the processing device to:
• extract semantic features in a semantic domain from semantic action labels, transform the semantic features from the semantic domain into mixed features in a mixed domain, and store the mixed features in a feature database;
• extract visual features in a visual domain from a video stream;
• determine if the visual features indicate an unseen action in the video stream;
• if no unseen action is determined, apply an offline classification model to the visual features to identify seen actions, assign identifiers to the identified seen actions, transform the visual features from the visual domain into mixed features in the mixed domain, and store the mixed features and seen action identifiers in the feature database;
• and
• if an unseen action is determined, transform the visual features from the visual domain into mixed features in the mixed domain, apply a continual learner model to mixed features from the feature database to identify unseen actions in the video stream, assign identifiers to the identified unseen actions, and store the unseen action identifiers in the feature database.

A method comprising:

• extracting semantic features in a semantic domain from semantic action labels, transforming the semantic features from the semantic domain into mixed features in a mixed domain, and storing the mixed features in a feature database;
• extracting visual features in a visual domain from a video stream;
• determining if the visual features indicate an unseen action in the video stream;
• if no unseen action is determined, applying an offline classification model to the visual features to identify seen actions, assigning identifiers to the identified seen actions, transforming the visual features from the visual domain into mixed features in the mixed domain, and storing the mixed features and seen action identifiers in the feature database;
• and
• if an unseen action is determined, transforming the visual features from the visual domain into mixed features in the mixed domain, applying a continual learner model to mixed features from the feature database to identify unseen actions in the video stream, assigning identifiers to the identified unseen actions, and storing the unseen action identifiers in the feature database.