Detecting facial expressions in digital images

An image processing method, comprising:

• identifying a group of pixels that correspond to a face within at least one digitally-acquired image on a portable image acquisition device using a face detector;
• acquiring a stream of images using the portable image acquisition device, where the stream of images comprises a collection of images having a lower resolution than the resolution of an image sensor of the portable image acquisition device;
• and
• performing in real time as the stream of images is acquired by the portable image acquisition device steps including:
• tracking pose of the face within the stream of images;
• for each interval of a plurality of intervals of the stream of images, determining a facial expression state based upon the tracked pose of the face by accumulating facial expression state information over multiple images of the interval and classifying a facial expression state for the interval based on the accumulated facial expression state information and logic that spans over several images of the interval, wherein each interval of the plurality of intervals includes one or more previously acquired images from the stream of images;
• determining a facial expression state for a first interval of the plurality of intervals;
• determining a facial expression state for a second interval of the plurality of intervals;
• detecting a facial expression state transition between the first interval and the second interval based upon the determined facial expression states for the first interval and the second interval, wherein the determined facial expression state for the first interval is different from the determined facial expression state for the second interval;
• in response to the detected transition between facial expression states, performing at least one image post-processing operation to at least one previously acquired image from the stream of images acquired by the portable image acquisition device;
• and
• causing display of the post-processed image on a display coupled to the portable image acquisition device.

A portable device, comprising:

• a camera;
• a display;
• a set of one or more processors capable of performing steps comprising:
• acquiring at least one image using the camera;
• identifying a group of pixels that correspond to a face within the at least one acquired image using a face detector;
• acquiring a stream of images using the camera, where the stream of images comprises a collection of images having a lower resolution than the resolution of an image sensor of the camera;
• and
• performing in real time as the stream of images is acquired steps comprising:
• tracking pose of the face within the stream of images;
• for each interval of a plurality of intervals of the stream of images, determining a facial expression state based upon the tracked pose of the face by accumulating facial expression state information over multiple images of the interval and classifying a facial expression state for the interval based on the accumulated facial expression state information and logic that spans over several images of the interval, wherein each interval of the plurality of intervals includes one or more previously acquired images from the streams of images;
• determining a facial expression state for a first interval of the plurality of intervals;
• determining a facial expression state for a second interval of the plurality of intervals;
• detecting a facial expression state transition between the first interval and the second interval based upon the determined facial expression states for the first interval and the second interval, wherein the determined facial expression state for the first interval is different from the determined facial expression state for the second interval;
• in response to the detected transition between facial expression states, performing at least one image post-processing operation to at least one previously acquired image from the stream of images acquired by the camera;
• and
• displaying the post-processed image via the display.