Method and device for dynamic recognition of emotion based on facial muscle movement monitoring

A method for dynamic recognition of emotion based on facial muscle movement monitoring, comprising:

• obtaining muscle movement data of a key area with a large amplitude of human facial expression movement on one side and continuous frame images of a key area with a large amplitude of human facial expression movement on the other side, and corresponding the muscle movement data and the continuous frame images to symmetrical key areas respectively;
• obtaining multiple emotional states related to emotions, associating each emotional state with its corresponding continuous frame images, associating each emotional state with the muscle movement data corresponding to the continuous frame images at the same location and at the same time, to form a training set, building an emotion recognition model through the training set;
• inputting the muscle movement data and continuous frame images obtained in real time into the emotion recognition model to obtain a corresponding emotional state, specifically comprising:
• splicing the continuous frame images corresponding to each different key area, and extracting image feature data of spliced images of the key areas;
• judging whether muscle changes have occurred in each key area based on changes in the image feature data;
• if changes occur in all key areas, determining muscle change feature data of each key area based on the image feature data, and determining the corresponding emotional state based on a correspondence relationship between the muscle change feature data and the emotional state;
• if one key area or multiple key areas have not changed, judging whether the muscle movement data at the same location and at the same time corresponding to a local area image that has not changed has changed:
• if it has changed, then converting the muscle movement data to corresponding muscle change feature data, and determining the corresponding emotional state based on the correspondence relationship between the muscle change feature data and the emotional state;
• wherein the step of “obtaining muscle movement data of a key area with a large amplitude of human facial expression movement on one side” comprises:
• attaching at least one three-dimensional displacement sensor to the key area with a large amplitude of human facial expression movement on the one side;
• and if the number of the three-dimensional displacement sensors is greater than 1, arranging one of the three-dimensional displacement sensors at a geometric center point of the key area with a large amplitude of human facial expression movement, and arranging the other three-dimensional displacement sensors on boundary lines of the key area with a large amplitude of human facial expression movement which has been reduced in the same proportion;
• determining the muscle movement data of the key area based on displacement change information collected by all three-dimensional displacement sensors in the same key area, wherein the muscle movement data comprises muscle movement distances corresponding to time and position.