Region-based stabilized face tracking
AssigneeSnap Inc.
InventorsChen Cao, Menglei Chai, Linjie Luo, Oliver Woodford
Life of this patent
Estimating the term…
Abstract
Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program and a method for accessing a set of images depicting at least a portion of a face. A set of facial regions of the face is identified, each facial region of the set of facial regions intersecting another facial region with at least one common vertex which is a member of a set of facial vertices. For each facial region of the set of facial regions, a weight formed from a set of region coefficients is generated. Based on the set of facial regions and the weight of each facial region of the set of facial regions, the face is tracked across the set of images.
Claims20
A method comprising:
- accessing a set of images depicting at least a portion of a face;
- identifying a set of facial regions of the face, each facial region of the set of facial regions intersecting another facial region with at least one common vertex which is a member of a set of facial vertices;
- for each facial region of the set of facial regions, generating a weight formed from a set of region coefficients;
- and
- based on the set of facial regions and the weight of each facial region of the set of facial regions, tracking the face across the set of images, the tracking comprising:
- solving for rigid pose parameters and expression parameters in alternating optimizations, such that during rigid pose optimization, expression parameters are fixed while the rigid pose parameters are determined and, during expression optimization, rigid pose parameter are fixed while the expression parameter are determined;
- and
- based on the rigid pose parameters and expression parameters, tracking a first facial region of the set of facial regions based on a first model and a second facial region of the set of facial regions based on a second model.
A system comprising:
- one or more processors;
- and
- a non-transitory processor-readable storage medium storing processor executable instructions that, when executed by the one or more processors cause the one or more processors to perform operations comprising:
- accessing a set of images depicting at least a portion of a face;
- identifying a set of facial regions of the face, each facial region of the set of facial regions intersecting another facial region with at least one common vertex which is a member of a set of facial vertices;
- for each facial region of the set of facial regions, generating a weight formed from a set of region coefficients;
- and
- based on the set of facial regions and the weight of each facial region of the set of facial regions, tracking the face across the set of images, the tracking comprising:
- solving for rigid pose parameters and expression parameters in alternating optimizations, such that during rigid pose optimization expression parameters are fixed while the rigid pose parameters are determined and, during expression optimization, rigid pose parameters are fixed while the expression parameters are determined;
- and
- based on the rigid pose parameter and expression parameters, tracking a first facial region of the set of facial regions based on a first model and a second facial region of the set of facial regions based on a second model.
A non-transitory processor-readable storage medium storing processor executable instructions that, when executed by a processor of a machine, cause the machine to perform operations comprising:
- accessing a set of images depicting at least a portion of a face;
- identifying a set of facial regions of the face, each facial region of the set of facial regions intersecting another facial region with at least one common vertex which is a member of a set of facial vertices;
- for each facial region of the set of facial regions, generating a weight formed from a set of region coefficients;
- and
- based on the set of facial regions and the weight of each facial region of the set of facial regions, tracking the face across the set of images, the tracking comprising:
- solving for rigid pose parameters and expression parameters in alternating optimizations, such that during rigid pose optimization, expression parameters are fixed while the rigid pose parameters are determined and, during expression optimization, rigid pose parameters are fixed while the expression parameters are determined;
- and
- based on the rigid pose parameters and expression parameters, tracking a first facial region of the set of facial regions based on a first model and a second facial region of the set of facial regions based on a second model.
Dependencies are inferred from the claim text.