Computer automated interactive activity recognition based on keypoint detection

A computer system for interactive activity recognition, comprising:

• one or more processors, one or more computer-readable memories, one or more computer-readable tangible storage devices, and program instructions stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, wherein the computer system is capable of performing a method comprising:
• retrieving, by one or more processors, a temporal sequence of image frames from a video recording;
• identifying, by the one or more processors, first keypoints in each of the image frames in the temporal sequence, the first keypoints are associated with an object in the temporal sequence of image frames;
• identifying, by the one or more processors, second keypoints in each of the image frames in the temporal sequence, the second keypoints are associated with an individual interacting with the object;
• combining, by the one or more processors, the first keypoints with the second keypoints;
• extracting, by the one or more processors, spatial-temporal features from the combined first keypoints and second keypoints;
• and
• based on the extracted spatial-temporal features, training, by the one or more processors, a classification model for recognition of interactive activities between the individual and the object.

A computer program product for interactive activity recognition, comprising:

• one or more computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media, the program instructions comprising:
• program instructions to retrieve, by one or more processors, a temporal sequence of image frames from a video recording;
• program instructions to identify, by the one or more processors, first keypoints in each of the image frames in the temporal sequence, the first keypoints are associated with an object in the temporal sequence of image frames;
• program instructions to identify, by the one or more processors, second keypoints in each of the image frames in the temporal sequence, the second keypoints are associated with an individual interacting with the object;
• program instructions to combine, by the one or more processors, the first keypoints with the second keypoints;
• program instructions to extract, by the one or more processors, spatial-temporal features from the combined first keypoints and second keypoints;
• and
• based on the extracted spatial-temporal features, program instructions to train, by the one or more processors, a classification model for recognition of interactive activities between the individual and the object.