Jaw tracking without markers for facial performance capture

A method, comprising:

• obtaining, at a computing device, a trained model that provides a mapping from facial skin motion to jaw motion, the trained model created using facial training data that includes, for each of one or more subjects, a facial skin geometry and corresponding jaw pose tracked over a plurality of times;
• determining, at the computing device, a first facial skin motion of a first subject corresponding to a facial performance capture, the first subject being different from the one or more subjects;
• predicting, at the computing device, using the trained model, based at least on the first facial skin motion, a first jaw motion of a jaw of the first subject, the first jaw motion corresponding to the facial performance capture;
• and
• generating, at the computing device, using the first jaw motion predicted using the trained model, a facial animation of a digital character.

A non-transitory computer-readable medium having executable instructions stored thereon that, when executed by a processor, cause a system to perform operations comprising:

• obtaining a trained model that provides a mapping from facial skin motion to jaw motion, the trained model created using facial training data captured for one or more subjects;
• determining a first facial skin motion of a first subject corresponding to a facial performance capture, the first subject being different from the one or more subjects;
• predicting, using the trained model, based at least on the first facial skin motion, a first jaw motion of a jaw of the first subject, the first jaw motion corresponding to the facial performance capture;
• and
• generating, using the first jaw motion predicted using the trained model, a facial animation of a digital character,
• wherein determining the first facial skin motion comprises determining multiple skin features corresponding to the first facial skin motion, and determining the multiple skin features comprises determining a position of multiple skin feature vertices relative to a skull of the first subject.

A method, comprising:

• obtaining, at a computing device, a trained model that provides a mapping from facial skin motion to jaw motion, the trained model created using facial training data captured for one or more subjects;
• determining, at the computing device, a first facial skin motion of a first subject corresponding to a facial performance capture, the first subject being different from the one or more subjects;
• predicting, at the computing device, using the trained model, based at least on the first facial skin motion, a first jaw motion of a jaw of the first subject, the first jaw motion corresponding to the facial performance capture;
• and
• generating, at the computing device, using the first jaw motion predicted using the trained model, a facial animation of a digital character,
• wherein determining the first facial skin motion comprises determining multiple skin features corresponding to the first facial skin motion, and determining the multiple skin features comprises determining a position of multiple skin feature vertices relative to a skull of the first subject.