Methods and systems for markerless facial motion capture

A method for training an artificial intelligence engine operable to convert a markerless digital image of a target face into a mapping comprising a digital representation of the target face, the method comprising:

• preparing a training data set for the artificial intelligence engine, wherein preparing the training data set comprises:
• obtaining, using one or more first image capture devices, markered digital images of the target face of a first person;
• obtaining, using one or more second image capture devices, markerless digital images of one or more skin parts;
• obtaining, using one or more third image capture devices, markered digital images of the one or more skin parts;
• training an intermediate artificial intelligence engine embodied by one or more processors to obtain a trained intermediate artificial intelligence engine operable to remove markers from markered digital images of skin, wherein training the intermediate artificial intelligence engine comprises:
• providing the markered digital images of the one or more skin parts to the one or more processors as a source training data set for the intermediate artificial intelligence engine;
• and
• providing the markerless digital images of the one or more skin parts to the one or more processors as a target training data set for the intermediate artificial intelligence engine;
• employing the trained intermediate artificial intelligence engine embodied by the one or more processors to remove markers from each of the markered digital images of the target face of the first person to create, for each markered digital image, a corresponding synthetic markerless digital image of the target face of the first person;
• and
• converting each markered digital image of the target face of the first person into a training mapping to create a corresponding training mapping for each synthetic markerless digital image;
• and
• providing the training data set to the artificial intelligence engine, the training data set comprising labelled pairs of the synthetic markerless digital images of the target face of the first person and the corresponding training mappings;
• training the artificial intelligence engine to obtain a trained artificial intelligence engine operable to convert a markerless digital image of the target face into a mapping comprising a digital representation of the target face, wherein training the artificial intelligence engine comprises:
• providing the synthetic markerless digital images to the artificial intelligence engine as a source training data set;
• providing the corresponding training mappings to the artificial intelligence engine as a target training data set.

A method of creating an animation based on a plurality of markerless target digital images of a target face, the method comprising:

• preparing a training data set for an artificial intelligence engine, wherein preparing the training data set comprises:
• obtaining, using one or more first image capture devices, markered digital images of the target face of a first person;
• obtaining, using one or more second image capture devices, markerless digital images of one or more skin parts;
• obtaining, using one or more third image capture devices, markered digital images of the one or more skin parts;
• training an intermediate artificial intelligence engine embodied by one or more processors to obtain a trained intermediate artificial intelligence engine operable to remove markers from markered digital images of skin, wherein training the intermediate artificial intelligence engine comprises:
• providing the markered digital images of the one or more skin parts to the one or more processors as a source training data set for the intermediate artificial intelligence engine;
• and
• providing the markerless digital images of the one or more skin parts to the one or more processors as a target training data set for the intermediate artificial intelligence engine;
• employing the trained intermediate artificial intelligence engine embodied by the one or more processors to remove markers from each of the markered digital images of the target face of the first person to create, for each markered digital image, a corresponding synthetic markerless digital image of the target face of the first person;
• and
• converting each markered digital image of the target face of the first person into a training mapping to create a corresponding training mapping for each synthetic markerless digital image;
• and
• providing the training data set to the artificial intelligence engine, the training data set comprising labelled pairs of the synthetic markerless digital images of the target face of the first person as source training data and the corresponding training mappings as target training data;
• training, using the training data set, the artificial intelligence engine to convert markerless digital image of the target face into mappings comprising digital representations of the target face;
• converting, using the trained artificial intelligence engine, the plurality of markerless target digital images of the target face into a plurality of corresponding target mappings, each comprising a digital representation of the target face;
• creating an animation wherein each frame of the animation is based at least in part on one of the plurality of corresponding target mappings.

A system for preparing a training data set for an artificial intelligence engine operable to convert a markerless digital image of a target face into a mapping comprising a digital representation of the target face, the system comprises:

• one or more first image capture devices for obtaining markered digital images of the target face of a first person;
• one or more second image capture devices for obtaining markerless digital images of one or more skin parts;
• one or more third image capture devices for obtaining markered digital images of the one or more skin parts;
• a processor connected to receive, from the one or more first, second and third image capture devices, the markered digital images of the target face of a first person, the markerless digital images of one or more skin parts and the markered digital images of the one or more skin parts;
• the processor configured to:
• train an intermediate artificial intelligence engine embodied by the processor to obtain a trained intermediate artificial intelligence engine operable to remove markers from markered digital images of skin, wherein training the intermediate artificial intelligence engine comprises:
• providing the markered digital images of the one or more skin parts to the processor as a source training data set for the intermediate artificial intelligence engine;
• and
• providing the markerless digital images of the one or more skin parts to the processor as a target training data set for the intermediate artificial intelligence engine;
• employ the trained intermediate artificial intelligence engine embodied by the processor to remove markers from each of the markered digital images of the target face of the first person to create, for each markered digital image, a corresponding one of the synthetic markerless digital images of the target face of the first person;
• convert each markered digital image of the target face of the first person into a training mapping to create a corresponding training mapping for each synthetic markerless digital image;
• and
• provide a training data set to the artificial intelligence engine, the training data set comprising labelled pairs of the synthetic markerless digital images of the target face of the first person and the corresponding training mappings to thereby train the artificial intelligence engine to convert markerless images of the target face into a mapping comprising a digital representation of the target face.