System and method for reconstruction of a three-dimensional human head model from a single image

A method for reconstructing a three-dimensional (3D) face model from an input 2D image of the face, the method comprising:

• feeding the input 2D image into at least one preparatory network to generate 3D landmarks, facial features and segmentation maps;
• feeding the 3D landmarks, the facial features, and the segmentation maps to a parameter regressor network to predict parameters of a 3D face parametric model, camera parameters, hair parameters and wrinkle parameters;
• generating an initial UV texture map using the input 2D image and the parameters of the 3D face parametric model, the camera parameters, the hair parameters, and the wrinkle parameters;
• feeding the initial UV texture map into a UV completion network to generate a full UV texture map and illumination parameters, for completing areas that are missing in the initial UV texture map;
• and
• generating the 3D face model using the parameters of the 3D face parametric model, the camera parameters, the hair parameters, the wrinkle parameters, the full UV texture map, and the illumination parameters,
• wherein the parameter regressor network and the UV completion network are jointly trained using, as the input 2D image, a synthetic 2D image that is generated from a set of synthetic parameters of the 3D face parametric model, synthetic camera parameters, synthetic hair parameters and synthetic wrinkle parameters.

A system for reconstructing a three-dimensional (3D) face model from an input 2D image of the face, the system comprising:

• a memory;
• and
• a processor configured to:
• feed the input 2D image into at least one preparatory network to generate image features;
• feed the image features to a parameter regressor network (M1) to predict at least one of:
• parameters of a 3D face parametric model, camera parameters, hair parameters and wrinkle parameters;
• generate an initial UV texture map using the input 2D image and the at least one of the parameters of the 3D face parametric model, the camera parameters, the hair parameters or the wrinkle parameters;
• feed the initial UV texture map into a UV completion network to generate a full UV texture map and illumination parameters for completing areas that are missing in the initial UV texture map;
• and
• generate the 3D face model using the parameters of the 3D face parametric model, the camera parameters, the hair parameters, the wrinkle parameters, the full UV texture map and the illumination parameters,
• wherein the parameter regressor network and the UV completion network are jointly trained using, as the input 2D image, a synthetic 2D image that is generated from a set of synthetic parameters.

The system of claim 20, wherein the loss function comprises a combination of at least one of:

• a reconstruction loss calculated between the input 2D image and the rendered 2D image,
• a perceptual loss calculated between the input 2D image and the rendered 2D image,
• a regularization loss calculated on the shape parameters and the expression parameters,
• a consistency loss calculated on the shape parameters of input 2D mages of a single person taken from different angles, and
• a consistency loss calculated on the full UV texture maps of input 2D mages of a single person taken from different angles.