Systems and methods for generating facial expressions in a user interface

A method of generating an avatar-based user interface for a computing device, comprising:

• capturing image data of a user over time using a digital camera;
• generating a plurality of facial expression descriptor vectors based on the image data;
• using a first conditional deep convolutional generative adversarial network (DC-GAN) model to generate an expressive facial sketch image of an avatar based on the plurality of facial expression descriptor vectors;
• using second conditional DC-GAN model to generate a facial expression image from the expressive facial sketch image;
• displaying the facial expression image as a user interface avatar on a display device;
• and
• training the first and second DC-GAN models based on a dataset of facial images of a pair of participants in a dyadic interaction.

A system for generating facial expressions in an avatar-based user interface for a computing device, comprising:

• a digital camera configured to capture image data of a user over time;
• a user interface application, stored on a non-transitory computer-readable medium and executable by a processor, the user interface application including,
• an expression feature extractor configured to process the image data of the user to generate a plurality of facial expression descriptor vectors, wherein the plurality of facial expression descriptor vectors represent likelihoods of a plurality of expression classes being present in the image data of the user,
• a sketch generation module configured to use a first conditional deep convolutional generative adversarial network (DC-GAN) model to generate an expressive facial sketch image of an avatar based on the plurality of facial expression descriptor vectors, and
• an image generation module configured to use a second conditional DC-GAN model to generate a facial expression image from the expressive facial sketch image;
• and
• a display device configured to display the facial expression image as a user interface avatar.