US 2023/0144458 A1 · ESTIMATING FACIAL EXPRESSIONS USING FACIAL LANDMARKS

Abstract

In examples, locations of facial landmarks may be applied to one or more machine learning models (MLMs) to generate output data indicating profiles corresponding to facial expressions, such as facial action coding system (FACS) values. The output data may be used to determine geometry of a model. For example, video frames depicting one or more faces may be analyzed to determine the locations. The facial landmarks may be normalized, then be applied to the MLM(s) to infer the profile(s), which may then be used to animate the mode for expression retargeting from the video. The MLM(s) may include sub-networks that each analyze a set of input data corresponding to a region of the face to determine profiles that correspond to the region. The profiles from the sub-networks, along global locations of facial landmarks may be used by a subsequent network to infer the profiles for the overall face.

Claims20

Dependent claims

A method comprising:

determining, using image data representing one or more images depicting one or more faces, location data indicating one or more locations of one or more facial landmarks corresponding to the one or more faces;
applying, using the location data, the one or more locations of the one or more facial landmarks to one or more machine learning models (MLMs) to generate output data indicating one or more profiles corresponding to one or more facial expressions;
and
determining, using the output data, one or more models having geometry corresponding to the one or more profiles.

1 more element

A system comprising:

one or more processing units to perform operations including:
analyzing video data representative of one or more sequences of images depicting one or more faces to determine one or more locations of one or more facial landmarks corresponding to the one or more faces;
based at least on the analyzing, determining one or more profiles corresponding to one or more facial expressions using one or more machine learning models (MLMs) trained to infer the one or more profiles from at least the one or more locations;
and
generating an animation of one or more models based at least on the one or more profiles corresponding to the one or more facial expressions.

2 more elements

Dependencies are inferred from the claim text.

Estimating facial expressions using facial landmarks

Life of this patent

Estimating facial expressions using facial landmarks

Life of this patent

Abstract

Claims20