Facial expressions from eye-tracking cameras
AssigneeMagic Leap, Inc.
InventorsAlexander Ilic, Martin Georg Zahnert
Life of this patent
Estimating the term…
Abstract
Systems and methods for synthesizing an image of the face by a head-mounted display (HMD), such as an augmented reality device, are disclosed. The HMD may be able to observe only a portion of the face with an inward-facing imaging system, e.g., the periocular region. The systems and methods described herein can generate a mapping of a conformation of the portion of the face that is not imaged based at least partly on a conformation of the portion of the face that is imaged. The HMD can receive an image of a portion of the face and use the mapping to determine a conformation of the portion of the face that is not observed. The HMD can combine the observed and unobserved portions to synthesize a full face image.
Claims20
A wearable system configured to personalize a mapping indicative of correlations between periocular features and lower face parameters of a wearer of the wearable system, the wearable system comprising:
- an outward-facing imaging system configured to capture images of at least an unobstructed region of the wearer's face in a mirror;
- and
- a hardware processor programmed to:
- instruct the wearer to mimic a requested expression;
- acquire a mirror image of the unobstructed region of the wearer's face via the outward-facing imaging system;
- detect one or more facial features in the mirror image;
- in response to determining that the detected one or more facial features match a corresponding one or more expected facial features associated with the requested expression, extract one or more key features of the wearer from the acquired mirror image;
- determine, based at least on the extracted one or more key features, a wearer-specific lower face conformation;
- and
- calculate wearer-specific adjustments to the mapping based at least on comparison of the wearer-specific lower face conformation and an expected lower face conformation.
A method for personalizing a mapping indicative of correlations between periocular features and lower face parameters of a wearer of a wearable system comprising an outward-facing imaging system, the method comprising:
- instructing the wearer to mimic a requested expression;
- acquiring a mirror image of at least an unobstructed region of the wearer's face via the outward-facing imaging system;
- detecting one or more facial features in the mirror image;
- in response to determining that the detected one or more facial features match a corresponding one or more expected facial features associated with the requested expression, extracting one or more key features of the wearer from the acquired mirror image;
- determining, based at least on the extracted one or more key features, a wearer-specific lower face conformation;
- and
- calculating wearer-specific adjustments to the mapping based at least on comparison of the wearer-specific lower face conformation and an expected lower face conformation.
Dependencies are inferred from the claim text.