Camera Reprojection for Faces

One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

• receive an image of a portion of a face of a first user, wherein the image is captured by a camera coupled to an artificial-reality headset worn by the first user;
• access a three-dimensional (3D) facial model representative of the face of the first user;
• identify one or more facial features captured in the image;
• determine a relative pose of the camera relative to the 3D facial model based on the identified one or more facial features in the image and predetermined feature locations on the 3D facial model;
• determine a mapping relationship between the image and the 3D facial model by projecting the image of the portion of the face of the first user onto the 3D facial model from the relative pose of the camera;
• and
• cause an output image of a facial representation of the first user to be rendered using at least the 3D facial model and the mapping relationship between the image and the 3D facial model.

A system comprising:

• one or more processors;
• and
• one or more computer-readable non-transitory storage media coupled to one or more of the processors and comprising instructions operable when executed by one or more of the processors to cause the system to:
• receive an image of a portion of a face of a first user, wherein the image is captured by a camera coupled to an artificial-reality headset worn by the first user;
• access a three-dimensional (3D) facial model representative of the face of the first user;
• identify one or more facial features captured in the image;
• determine a relative pose of the camera relative to the 3D facial model based on the identified one or more facial features in the image and predetermined feature locations on the 3D facial model;
• determine a mapping relationship between the image and the 3D facial model by projecting the image of the portion of the face of the first user onto the 3D facial model from the relative pose of the camera;
• and
• cause an output image of a facial representation of the first user to be rendered using at least the 3D facial model and the mapping relationship between the image and the 3D facial model.