System and method for recognition and annotation of facial expressions

A computer-implemented method for analyzing an image to determine AU and AU intensity using color features in the image, comprising:

• identifying changes defining transition of an AU from inactive to active, wherein the changes are selected from the group consisting of chromaticity, hue and saturation, and luminance;
• and
• applying a Gabor transform to the identified color changes to gain invariance to the timing of this change during a facial expression.

A method, comprising:

• identifying changes defining transition of an emotion from not present to present are color transitions in a face in a video sequence resulting from blood flow in the face.

A computer-implemented method for analyzing an image to determine AU and AU intensity in face images, comprising:

• training a deep neural network with a set of training images having faces, wherein the deep neural network is trained to identify AU in the face images;
• and
• identifying local loss and global loss with the deep neural network in the face images to determine AUs in the face images.

A method, comprising:

• determining a neutral face in an image, the image having color;
• modifying the color of the image of the neutral face to yield the appearance of a determined expression of an emotion or an AU.

A method, comprising:

• determining an emotion or AU in a face in an image, the image having color, the emotion and AU having intensities;
• modifying the color of the image to increase or decrease the intensity of the emotion or AU to change the perception of the emotion or the AU.