Real-time avatars using dynamic textures

A method of generating real-time facial avatars comprising:

• employing a generative network within the generative adversarial neural network to iteratively create facial textures;
• employing a discriminator network within the generative adversarial neural network to iteratively approve or refuse the facial textures as adequately corresponding to a training data set or not adequately corresponding to the training data set;
• applying the generative adversarial neural network to create a set of predetermined key expressions after training is complete;
• receiving a single image of a human face in a neutral facial pose;
• generating a morphable three-dimensional model of the human face and a texture map based upon the single image;
• generating a set of created images for the human face from the texture map, each of the created images representing a predetermined facial expression;
• generating blendshape coefficients and texture maps for each of the set of created images to thereby create the set of predetermined key expressions;
• and
• generating a series of facial action coding system (FACS) texture maps corresponding to a predetermined set of FACS expressions and an activation mask comprising a per-vertex deformation relative to the neutral facial pose for each desired FACS based upon the predetermined key expressions.

An apparatus comprising a storage medium storing a program having instructions which when executed by a processor will cause the processor to:

• train a generative adversarial neural network for application in creating a set of predetermined key expressions for an input image by causing the processor to:
• employ a generative network within the generative adversarial neural network to iteratively create facial textures;
• employ a discriminator network within the generative adversarial neural network to iteratively approve or refuse the facial textures as adequately corresponding to a training data set or not adequately corresponding to the training data set;
• and
• apply the generative adversarial neural network to create the set of predetermined key expressions after training is complete;
• and
• generate a facial avatar by causing the processor to:
• receive a single image of a human face in a neutral facial pose;
• generate a morphable three-dimensional model of the human face and a texture map based upon the single image;
• generate a set of created images for the human face from the texture map, each of the created images representing a predetermined facial expression;
• generate blendshape coefficients and texture maps for each of the set of created images to thereby create the set of predetermined key expressions;
• and
• generate a series of facial action coding system (FACS) texture maps corresponding to a predetermined set of FACS expressions and an activation mask comprising a per-vertex deformation relative to the neutral facial pose for each desired FACS based upon the predetermined key expressions.