Selecting combination of parameters for preprocessing facial images of wearer of head-mountable display

A method comprising:

• capturing, using one or multiple cameras of a head-mountable display (HMD), a plurality of calibration facial images of a wearer of the HMD as the wearer exhibits calibration facial expressions;
• applying a preprocessing technique to each calibration facial image using a plurality of combinations of parameters to generate a plurality of preprocessed calibration facial images respectively corresponding to the combinations;
• applying a machine learning model to each preprocessed calibration facial image to predict facial action units corresponding to the calibration facial expression exhibited by the wearer in the calibration facial image corresponding to the preprocessed calibration facial image;
• retargeting the facial action units predicted from each preprocessed calibration facial image onto an avatar to render one of a plurality of avatar facial images in which the avatar exhibits the calibration facial expression exhibited by the wearer in the calibration facial image corresponding to the preprocessed calibration facial image;
• detecting wearer facial landmarks within each calibration facial image according to a specified facial model;
• detecting avatar facial landmarks within each avatar facial image according to the specified facial model and that correspond to the wearer facial landmarks detected within the calibration facial image corresponding to the avatar facial image;
• and
• selecting the combination of parameters yielding a highest similarity between the avatar facial landmarks and the wearer facial landmarks corresponding to the avatar facial landmarks.

A system comprising:

• a head-mountable display (HMD) having one or multiple cameras to capture calibration facial images of a wearer of the HMD as the wearer exhibits calibration facial expressions;
• a processor;
• and
• a memory storing program code executable by the processor to:
• apply a preprocessing technique to each calibration facial image using a plurality of combinations of parameters to generate preprocessed facial images respectively corresponding to the combinations;
• apply a machine learning model to each preprocessed facial image to predict facial action units;
• retarget the facial action units predicted from each preprocessed facial image onto an avatar to render one of a plurality of avatar facial images;
• detect avatar facial landmarks within each avatar facial image and wearer facial landmarks within each facial image;
• and
• select the combination of parameters yielding a highest similarity between the avatar facial landmarks and the wearer facial landmarks corresponding to the avatar facial landmarks.