Real-time facial restoration and relighting in videos using facialenhancement neural networks

A system comprising:

• an image restoration machine-learning model that includes:
• a distortion classifier;
• and
• an autoencoder model having an encoder and a generator;
• a processor;
• and
• a computer memory comprising instructions that, when executed by the processor, cause the system to carry out operations comprising:
• generating a distortion classification for a face in a digital image using the distortion classifier, wherein the distortion classifier predicts a degradation type in the digital image;
• generating encoded feature vectors of the face using the autoencoder model;
• creating a light-enhanced face image of the face with the autoencoder model using the distortion classification and the encoded feature vectors;
• and
• generating an enhanced digital image by combining the light-enhanced face image with an image background.

A computer-implemented method for generating a light-enhanced face image of a face within a digital image, comprising:

• generating a distortion classification for the face using a distortion classifier, wherein the distortion classifier predicts a degradation type in the digital image;
• generating encoded feature vectors of the face using an autoencoder model;
• generating the light-enhanced face image by combining the distortion classification with the encoded feature vectors generated;
• generating an enhanced digital image having a re-lit face by combining the light-enhanced face image with an image background, wherein the enhanced digital image improves lighting on the face as compared to the digital image and maintains a same or substantially similar image background as the digital image;
• and
• providing the enhanced digital image for display on a computing device.