Systems and methods to process images for skin analysis and to visualize skin analysis

A computer implemented method of skin diagnosis comprising:

• receiving an image, that is a selfie face image from a user mobile device;
• performing a plurality (k) of data augmentations to the image to yield k augmented images for processing;
• processing each of the k augmented images using a deep neural network to obtain k activation maps for a skin diagnosis, the deep neural network further configured to classify image pixels to determine the skin diagnosis for a skin condition, wherein the deep neural network, which is already trained using a dataset of facial image data comprising selfie face images from user mobile devices, is configured as a regressor and a classifier to determine the skin diagnosis as an integer value on a scale classifying a severity of the skin condition over the image, and is configured to provide an activation map to visualize the skin diagnosis;
• defining a final activation map for the image from the k activation maps to visualize the skin diagnosis;
• and
• visualizing the skin diagnosis in association with the image.

A computer program product comprising a non-transitory storage device storing instruction that, when executed by a processor of a skin diagnostic device, cause the skin diagnostic device to:

• receive an image, that is a selfie face image from a user mobile device;
• perform a plurality (k) of data augmentations to the image to yield k augmented images for processing;
• process each of the k augmented images using a deep neural network to obtain k activation maps for a skin diagnosis;
• and
• define a final activation map for the image from the k activation maps to visualize the skin diagnosis;
• and
• wherein the deep neural network is already trained using a dataset of facial image data comprising selfie face images from user mobile devices, and is configured:
• to classify image pixels to determine the skin diagnosis for a skin condition,
• to provide an activation map to visualize the skin diagnosis,
• as a regressor and a classifier to determine the skin diagnosis, and
• with auxiliary tasks to determine one or both of an ethnicity prediction and gender prediction.