Automated single viewpoint human action recognition by matching linked sequences of key poses

An automated human action recognition system for automatically recognizing one or more actions of a human from 2D input image data representing a sequential series of input images of the human performing the one or more actions from the same viewpoint, each input image being represented by 2D image data, the system comprising:

• a computer memory system that includes computer hardware and contains 2D reference image data representing a plurality of reference actions performed by a human, wherein
• the 2D reference image data includes a plurality of linked sequences of key poses, including a linked sequence of key poses for each reference action, and
• for each reference action:
• each key pose within the linked sequence of key poses for the reference action consists essentially of 2D image data that is representative of a human figure performing the reference action at a selected point during the reference action;
• timing of each selected point within the linked sequence of key poses for the reference action is based on changes in position of the human figure during the performance of the reference action;
• and
• the linked sequence of key poses for the reference action uniquely distinguish the reference action from the linked sequence of key poses for all of the other reference actions;
• and
• a computer processing system that includes computer hardware and that is configured to determine which of the linked sequences best matches the 2D input image data,
• wherein:
• the computer processing system is configured to automatically select each selected point by analyzing three-dimensional image data that is representative of a human figure performing each of the reference actions;
• the computer processing system is configured to correlate each selected point with a time when computed motion energy of the human figure during the performance of each reference action is at a peak or at a valley, and to cause the sequence of the selected points in each linked sequence for each reference action to alternate between a time when the computed motion energy is at a peak and a time when the computed motion energy is at a valley;
• one of the linked sequences includes a link between the key pose at the end of another linked sequence for a first of the reference actions to the key pose at the beginning of a still further linked sequence for a second of the reference actions;
• or
• the computer processing system is configured to determine which of the linked sequences best matches the 2D input image data by, in part, determining how well each input image matches each key pose, and to utilize a modified PMK algorithm to determine how well each input image matches each key pose.