Normalization of facial images using deep neural networks

A method of generating a normalization of a single two-dimensional image of an unconstrained human face comprising:

• receiving the single two-dimensional image of the unconstrained human face;
• using a perspective undistortion network to remove perspective distortion from the unconstrained human face to generate an undistorted face based on the unconstrained human face, wherein removing perspective distortion includes performing three-dimensional face fitting to the undistorted face to obtain a fitted three-dimensional mesh, spherical harmonics shading coefficients, and UV texture;
• using a lighting translation network to generate an evenly lit face based on the undistorted face by normalizing lighting of the undistorted face;
• and
• using an expression neutralization network to generate a frontalized and neutralized expression face based on the evenly lit face.

An apparatus comprising a storage medium storing a program having instructions which when executed by a processor will cause the processor to:

• receive the single two-dimensional image of an unconstrained human face;
• use a perspective undistortion network to remove perspective distortion from the unconstrained human face to generate an undistorted face based on the unconstrained human face, wherein removing perspective distortion includes performing three-dimensional face fitting to the undistorted face to obtain a fitted three-dimensional mesh, spherical harmonics shading coefficients, and UV texture;
• use a lighting translation network to generate an evenly lit face based on the undistorted face by normalizing lighting of the undistorted face;
• and
• use an expression neutralization network to generate a frontalized and neutralized expression face based on the evenly lit face.