Device, method and non-transitory recording medium for building a hidden semi-Markov model to estimate human action segments using hybrid learning

A partial action segment estimation model building method comprising:

• by a computer,
• in a hidden semi-Markov model including a plurality of second hidden Markov models each containing a plurality of first hidden Markov models using types of movement of a person as states, and the plurality of second hidden Markov models each using partial actions that are parts of actions defined by combining a plurality of the movements as states, learning observation probabilities for each of the movement types of the plurality of first hidden Markov models using unsupervised learning;
• fixing the learnt observation probabilities, augmenting input first supervised data so as to give second supervised data, and learning transition probabilities of the movements of the first hidden Markov models by supervised learning in which the second supervised data is employed;
• building the hidden semi-Markov model that is a model for estimating segments of the partial actions by using the learnt observation probabilities and the learnt transition probabilities;
• extracting feature vectors from sensor data generated by detecting postures of a person using sensors;
• taking a series of the extracted feature vectors as observation data, and estimate successive durations of respective action phases by comparison against the built HSMM;
• computing successive durations for the respective actions by adding the successive durations for the action phases contained in each action;
• and
• estimating time segments for each action from the successive durations of each action state, and the built HSMM is applied for estimating the time segments of the action that contains partial actions.

A non-transitory recording medium storing a program that causes a computer to execute a partial action segment estimation model building processing, the processing comprising:

• in a hidden semi-Markov model including a plurality of second hidden Markov models each containing a plurality of first hidden Markov models using types of movement of a person as states, and the plurality of second hidden Markov models each using partial actions that are parts of actions defined by combining a plurality of the movements as states, learning observation probabilities for each of the movement types of the plurality of first hidden Markov models using unsupervised learning;
• fixing the learnt observation probabilities, augmenting input first supervised data so as to give second supervised data, and learning transition probabilities of the movements of the first hidden Markov models by supervised learning in which the second supervised data is employed;
• building the hidden semi-Markov model that is a model for estimating segments of the partial actions by using the learnt observation probabilities and the learnt transition probabilities;
• extracting feature vectors from sensor data generated by detecting postures of a person using sensors;
• taking a series of the extracted feature vectors as observation data, and estimate successive durations of respective action phases by comparison against the built HSMM;
• computing successive durations for the respective actions by adding the successive durations for the action phases contained in each action;
• and
• estimating time segments for each action from the successive durations of each action state, and the built HSMM is applied for estimating the time segments of the action that contains partial actions.