Neural network based face detection and landmark localization

One or more non-transitory computer-readable media having a plurality of executable instructions embodied thereon, which, when executed by one or more processors, cause the one or more processors to perform a method comprising:

• jointly predicting, by a neural network, an adjusted bounding box of a face in an input image and initial facial landmark locations by outputting a representation of both the adjusted bounding box and the initial facial landmark locations from a common fully-connected layer of the neural network, each of the initial facial landmark locations corresponding to a two-dimensional point in the input image;
• mapping initial facial landmark locations to a 3D face model;
• determining a second set of predicted facial landmark locations in the input image from the 3D face model;
• identifying refined facial landmark locations in the input image by adjusting positions of the second set of predicted facial landmark locations;
• and
• causing presentation of a representation of the input image using the refined facial landmark locations.

A system comprising:

• a model manager configured to use one or more hardware processors to:
• use a joint calibration and alignment neural network to jointly predict (1) an adjusted bounding box of a face in an input image represented by a predicted scale change or offset vector for an initial bounding box of the face and (2) initial facial landmark locations of the face, each of the initial facial landmark locations corresponding to a two-dimensional point in the input image;
• and
• provide the initial facial landmark locations to a landmark location refiner configured to generate refined facial landmark locations in the input image from the initial facial landmark locations;
• and
• a presentation component configured to use the one or more hardware processors to cause presentation of a representation of the input image using the refined facial landmark locations.