Back to results
US 11,636,652 B2Granted
Periocular and audio synthesis of a full face image
AssigneeMagic Leap, Inc.
InventorAdrian Kaehler
Life of this patent
Estimating the term…
Abstract
Systems and methods for synthesizing an image of the face by a head-mounted device (HMD) are disclosed. The HMD may not be able to observe a portion of the face. The systems and methods described herein can generate a mapping from a conformation of the portion of the face that is not imaged to a conformation of the portion of the face observed. The HMD can receive an image of a portion of the face and use the mapping to determine a conformation of the portion of the face that is not observed. The HMD can combine the observed and unobserved portions to synthesize a full face image.
Claims19
A computing system comprising:
- a hardware processor programmed to:
- access an image of a periocular region of a user wherein features of a lower portion of the users' face are unobservable in the image;
- generate, based at least partly on the image of the periocular region of the user, periocular face parameters encoding a periocular conformation of at least the periocular region of the user;
- access a base model that was generated using images associated with a group of people not including the user;
- customize a mapping based at least in part on the base model and the image of the periocular region of the user,
- wherein an input of the mapping comprises at least the image of the periocular region of the user, and
- wherein an output of the mapping comprises lower face parameters that encode a conformation of the lower face of the user that are deduced from an analysis of the image of the periocular region of the user;
- apply the mapping to the image of the periocular region of the user to generate the lower face parameters;
- and
- combine the periocular face parameters and the lower face parameters to generate full face parameters associated with a three-dimensional (3D) face model.
7 more elements
A method comprising:
- accessing an image of the periocular region of a user acquired by an imaging system comprising one or more cameras positioned such that a periocular region of the user's face is observable by the imaging system and the user's lower face is unobservable by the imaging system;
- determining, based at least partly on the image, periocular face parameters encoding a periocular conformation of at least the periocular region of the user;
- accessing a base model that was generated using images associated with a group of people not including the user;
- customizing a mapping based at least in part on the base model and the image of the periocular region of the user,
- wherein an input of the mapping comprises at least the image of the periocular region of the user, and
- wherein an output of the mapping comprises lower face parameters that encode a conformation of the lower face of the user, and that are deduced from an analysis of the image of the periocular region of the user;
- applying the mapping to the image to generate the lower face parameters;
- combining the periocular face parameters and the lower face parameters to generate full face parameters associated with a three-dimensional (3D) face model;
- and
- generating a full face image based at least partly on the full face parameters.
7 more elements
Dependencies are inferred from the claim text.