# US 2026/0073731 A1 · Selecting combination of parameters for preprocessing facial images of wearer of head-mountable display

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0073731 A1
- Document ID: US20260073731A1
- Status: Published application
- Kind: A1
- Application number: 19/106,147
- Filed: 2022-08-31
- Published: 2026-03-12
- Earliest claimed: 2022-08-31
- Assignees: Hewlett-Packard Development Company, L.P.
- Applicants: Hewlett-Packard Development Company, L.P.
- Inventors: Shibo Zhang; Jishang Wei; Prahalathan Sundaramoorthy
- Web page: https://patentgrep.com/patent/US20260073731A1

## CPC

- G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships}
- G06V 40/174: {Facial expression recognition}
- G06T 13/40: of characters, e.g. humans, animals or virtual beings
- G06V 10/761: {Proximity, similarity or dissimilarity measures}
- G06V 10/82: using neural networks
- G06V 10/94: Hardware or software architectures specially adapted for image or video understanding
- G06V 40/166: {using acquisition arrangements}
- G06V 40/176: {Dynamic expression}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

For each facial image of a wearer of a head-mountable display (HMD), preprocessed facial images corresponding to combinations of preprocessing parameters are generated. A machine learning model is applied to each preprocessed facial image to predict facial action units. The facial action units predicted from each preprocessed facial image are retargeted onto an avatar to render an avatar facial image. Avatar facial landmarks within each avatar facial image and wearer facial landmarks within each facial image are detected. The combination of preprocessing parameters yielding a highest similarity between the avatar facial landmarks and the wearer facial landmarks corresponding to the avatar facial landmarks is selected.

## Claims

### Claim 1

A non-transitory computer-readable data storage medium storing program code executable by a processor to perform processing comprising:
for each of a plurality of facial images of a wearer of a head-mountable display (HMD), generating a plurality of preprocessed facial images respectively corresponding to combinations of preprocessing parameters;
applying a machine learning model to each preprocessed facial image to predict facial action units;
retargeting the facial action units predicted from each preprocessed facial image onto an avatar to render one of a plurality of avatar facial images;
detecting avatar facial landmarks within each avatar facial image and wearer facial landmarks within each facial image; and
selecting the combination of preprocessing parameters yielding a highest similarity between the avatar facial landmarks and the wearer facial landmarks corresponding to the avatar facial landmarks.

### Claim 2 (depends on 1)

The non-transitory computer-readable data storage medium of claim 1, wherein the processing further comprises:
preprocessing subsequent facial images of the wearer using the selected combination of preprocessing parameters;
applying the machine learning model to the preprocessed subsequent facial images to predict wearer facial action units for a facial expression of the wearer exhibited within the subsequent facial images; and
retargeting the wearer facial action units onto the avatar to render the avatar with the facial expression of the wearer for display.

### Claim 3 (depends on 2)

The non-transitory computer-readable data storage medium of claim 2, wherein the processing further comprises:
displaying the rendered avatar.

### Claim 4 (depends on 1)

The non-transitory computer-readable data storage medium of claim 1, wherein the processing further comprises:
instructing the wearer to exhibit specified different calibration facial expressions; and
capturing, using one or multiple cameras of the HMD, one or more of the facial images while the wearer is exhibiting each specified different calibration facial expression.

### Claim 5 (depends on 1)

The non-transitory computer-readable data storage medium of claim 1, wherein generating the preprocessed facial images for each facial image comprises:
independently applying a preprocessing technique to the facial image a plurality of times to generate the preprocessed facial images,
wherein a different combination of preprocessing parameters is used each time the preprocessing technique is applied.

### Claim 6 (depends on 1)

The non-transitory computer-readable data storage medium of claim 1, further comprising:
for each avatar facial image, calculating a similarity between the avatar facial landmarks detected within the avatar facial image and the wearer facial landmarks detected within the facial image corresponding to the avatar facial image; and
calculating a score for each combination of preprocessing parameters based on the similarity for each avatar facial image corresponding to the combination of preprocessing parameters,
wherein the combination of preprocessing parameters having a highest or lowest score is selected as the combination of preprocessing parameters yielding the highest similarity.

### Claim 7

A method comprising:
capturing, using one or multiple cameras of a head-mountable display (HMD), a plurality of calibration facial images of a wearer of the HMD as the wearer exhibits calibration facial expressions;
applying a preprocessing technique to each calibration facial image using a plurality of combinations of parameters to generate a plurality of preprocessed calibration facial images respectively corresponding to the combinations;
applying a machine learning model to each preprocessed calibration facial image to predict facial action units corresponding to the calibration facial expression exhibited by the wearer in the calibration facial image corresponding to the preprocessed calibration facial image;
retargeting the facial action units predicted from each preprocessed calibration facial image onto an avatar to render one of a plurality of avatar facial images in which the avatar exhibits the calibration facial expression exhibited by the wearer in the calibration facial image corresponding to the preprocessed calibration facial image;
detecting wearer facial landmarks within each calibration facial image according to a specified facial model;
detecting avatar facial landmarks within each avatar facial image according to the specified facial model and that correspond to the wearer facial landmarks detected within the calibration facial image corresponding to the avatar facial image; and
selecting the combination of parameters yielding a highest similarity between the avatar facial landmarks and the wearer facial landmarks corresponding to the avatar facial landmarks.

### Claim 8 (depends on 7)

The method of claim 7, further comprising:
capturing, using the cameras of the HMD, facial images as the wearer exhibits a facial expression;
applying the preprocessing technique to the captured facial images using the selected combination of parameters;
applying the machine learning model to the captured facial images as have been preprocessed to predict wearer facial action units for the facial expression exhibited by the wearer within the captured facial images; and
retargeting the wearer facial action units onto the avatar to render the avatar with the facial expression of the wearer for display.

### Claim 9 (depends on 7)

The method of claim 7, wherein the calibration facial expressions comprise a neutral facial expression, a left-smile facial expression, and a right-smile facial expression,
and wherein one or more of the calibration facial images are captured for each calibration facial expression.

### Claim 10 (depends on 7)

The method of claim 7, wherein the preprocessing technique comprises contrast-limited adaptive histogram equalization,
and wherein the preprocessing parameters comprise clip limit and grid size.

### Claim 11 (depends on 7)

The method of claim 7, further comprising:
for each avatar facial image, calculating a similarity between the avatar facial landmarks detected within the avatar facial image and the wearer facial landmarks detected within the calibration facial image corresponding to the avatar facial image; and
calculating a score for each combination of parameters based on the similarity for each avatar facial image corresponding to the combination of parameters,
wherein the combination of parameters having a highest or lowest score is selected as the combination of parameters yielding the highest similarity.

### Claim 12

A system comprising:
a head-mountable display (HMD) having one or multiple cameras to capture calibration facial images of a wearer of the HMD as the wearer exhibits calibration facial expressions;
a processor; and
a memory storing program code executable by the processor to:
apply a preprocessing technique to each calibration facial image using a plurality of combinations of parameters to generate preprocessed facial images respectively corresponding to the combinations;
apply a machine learning model to each preprocessed facial image to predict facial action units;
retarget the facial action units predicted from each preprocessed facial image onto an avatar to render one of a plurality of avatar facial images;
detect avatar facial landmarks within each avatar facial image and wearer facial landmarks within each facial image; and
select the combination of parameters yielding a highest similarity between the avatar facial landmarks and the wearer facial landmarks corresponding to the avatar facial landmarks.

### Claim 13 (depends on 12)

The system of claim 12, wherein the program code is executable by the processor to further:
apply the preprocessing technique to subsequently captured facial images of the wearer using the selected combination of parameters;
apply the machine learning model to the subsequently captured facial images as have been preprocessed to predict wearer facial action units for a facial expression exhibited by the wearer within the captured facial images; and
retarget the wearer facial action units onto the avatar to render the avatar with the facial expression of the wearer for display.

### Claim 14 (depends on 12)

The system of claim 12, wherein the preprocessing technique comprises contrast-limited adaptive histogram equalization,
and wherein the preprocessing parameters comprise clip limit and grid size.

### Claim 15 (depends on 12)

The system of claim 12, wherein the program code is executable by the processor to further:
for each avatar facial image, calculate a similarity between the avatar facial landmarks detected within the avatar facial image and the wearer facial landmarks detected within the calibration facial image corresponding to the avatar facial image; and
calculating a score for each combination of parameters based on the similarity for each avatar facial image corresponding to the combination of parameters,
wherein the combination of parameters having a highest or lowest score is selected as the combination of parameters yielding the highest similarity.
