Action classification based on manipulated object movement

A method for use with a computing device, the method comprising:

• receiving a first video including a plurality of frames;
• for each frame of the plurality of frames:
• determining that a target region of the frame includes a target object;
• determining a surrounding region within which the target region is located, wherein the surrounding region is smaller than the frame and the target region is smaller than the surrounding region;
• extracting one or more features located in the surrounding region;
• and
• from the one or more features, generating one or more manipulated object identifiers;
• for each of a plurality of pairs of frames of the first video respectively including a first frame and a second frame, determining a respective manipulated object movement between a first manipulated object identifier of the first frame and a second manipulated object identifier of the second frame;
• and
• classifying at least one action performed in the first video based on the manipulated object movements.

A computing device comprising:

• a processor configured to:
• receive a first video including a plurality of frames;
• for each frame of the plurality of frames:
• determine that a first target region of the frame includes a first hand and a second target region of the frame includes a second hand;
• determine a first surrounding region within which the first target region is located and a second surrounding region within which the second target region is located, wherein the first surrounding region and the second surrounding region are each smaller than the frame;
• identify one or more first surrounding region features located in the first surrounding region;
• identify one or more second surrounding region features located in the second surrounding region;
• and
• from the one or more first surrounding region features and/or the one or more second surrounding region features, generate one or more manipulated object identifiers that respectively identify one or more manipulated objects manipulated by the first hand and/or the second hand;
• for each of a plurality of pairs of frames of the first video respectively including a first frame and a second frame, determine a respective manipulated object movement between a first manipulated object identifier of the first frame and a second manipulated object identifier of the second frame;
• classify a plurality of actions performed in the first video based on the manipulated object movements.