# US 2025/0166263 A1 · Method and apparatus with blind face restoration

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0166263 A1
- Document ID: US20250166263A1
- Status: Published application
- Kind: A1
- Application number: 18/947,278
- Filed: 2024-11-14
- Published: 2025-05-22
- Earliest claimed: 2023-11-17
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Hui LI
- Web page: https://patentgrep.com/patent/US20250166263A1

## CPC

- G06T 3/40: Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 11/60: Creating or editing images; Combining images with text
- G06V 10/806: {of extracted features}
- G06V 10/82: using neural networks
- G06V 40/168: {Feature extraction; Face representation}
- G06T 2207/10024: Color image
- G06T 2207/20016: Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
- G06T 2207/20221: Image fusion; Image merging
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A processor-implemented method including obtaining a pyramid feature of an input face image, generating a first initial style feature corresponding to a highest level feature based on the highest level feature of the pyramid feature, and a latent code and noise of the input face image, generating a first style feature corresponding to each level feature based on an integrated feature corresponding to each level feature of the pyramid feature, the noise, and the latent code, and generating a high-quality face image of the input face image based on the first initial style feature and the first style feature corresponding to each level feature of the pyramid feature, the integrated feature corresponding to each level feature is generated based on each level feature, the first style feature corresponding to a higher level feature of each level feature, the latent code, and the noise and an integrated feature corresponding to the highest level feature of the pyramid feature is generated based on the highest level feature, the latent code, the noise, and the first initial style feature.

## Claims

### Claim 1

A processor-implemented method, the method comprising:
obtaining a pyramid feature of an input face image;
generating a first initial style feature corresponding to a highest level feature based on the highest level feature of the pyramid feature, and a latent code and noise of the input face image;
generating a first style feature corresponding to each level feature based on an integrated feature corresponding to each level feature of the pyramid feature, the noise, and the latent code; and
generating a high-quality face image of the input face image based on the first initial style feature and the first style feature corresponding to each level feature of the pyramid feature,
wherein the integrated feature corresponding to each level feature is generated based on each level feature, the first style feature corresponding to a higher level feature of each level feature, the latent code, and the noise, and
wherein an integrated feature corresponding to the highest level feature of the pyramid feature is generated based on the highest level feature, the latent code, the noise, and the first initial style feature.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
generating a second style feature corresponding to the higher level feature based on the first style feature corresponding to the higher level feature, the noise, and the latent code;
generating the integrated feature corresponding to each level feature based on the second style feature corresponding to the higher level feature, and each level feature; and
generating a second initial style feature corresponding to the highest level feature based on the first initial style feature, the latent code, and the noise, and generating the integrated feature corresponding to the highest level feature based on the second initial style feature and the highest level feature.

### Claim 3 (depends on 2)

The method of claim 2, wherein the generating of the integrated feature corresponding to each level feature based on the second style feature corresponding to the higher level feature, and each level feature comprises:
generating a first fully connected feature of the second style feature corresponding to the higher level feature and a first fully connected feature of each level feature by performing a full connection for each of the second style feature corresponding to the higher level feature, and each level feature;
generating a first feature by performing first weighted summation of the first fully connected feature of the second style feature corresponding to the higher level feature and the first fully connected feature of each level feature;
generating a second feature by performing the full connection for the first feature; and
generating the integrated feature corresponding to each level feature based on the second feature, and
wherein the generating of the integrated feature corresponding to the highest level feature based on the second initial style feature and the highest level feature comprises:
generating a first fully connected feature of the second initial style feature and a first fully connected feature of the highest level feature by performing the full connection for each of the second initial style feature and the highest level feature;
generating a third feature by performing second weighted summation of the first fully connected feature of the second initial style feature and the first fully connected feature of the highest level feature;
generating a fourth feature by performing the full connection for the third feature; and
obtaining the integrated feature corresponding to the highest level feature based on the fourth feature.

### Claim 4 (depends on 3)

The method of claim 3, wherein a first weight of the first weighted summation for generating the first feature is obtained based on the second style feature corresponding to the higher level feature, and
wherein a second weight of the second weighted summation for generating the third feature is obtained based on the second initial style feature.

### Claim 5 (depends on 4)

The method of claim 4, further comprising:
generating the first weight by performing a softmax operation for the first fully connected feature corresponding to the second style feature corresponding to the higher level feature; and
generating the second weight by performing the softmax operation for a second fully connected feature corresponding to the second initial style feature.

### Claim 6 (depends on 1)

The method of claim 1, wherein the generating of the high-quality face image of the input face image based on the first initial style feature and the first style feature corresponding to each level feature of the pyramid feature comprises:
obtaining a red, green, blue (RGB) image corresponding to each level feature based on the first style feature corresponding to each level feature of the pyramid feature, and obtaining an initial RGB image corresponding to the highest level feature based on the first initial style feature; and
generating a cumulative RGB image corresponding to each level feature by summing the RGB image corresponding to each level feature and an upscaled RGB image corresponding to the higher level feature of each level feature,
wherein a cumulative RGB image corresponding to a lowest level feature is a high-quality image,
wherein the upscaled RGB image corresponding to each level feature is generating by upscaling the cumulative RGB image corresponding to each level feature, and
wherein a cumulative RGB image corresponding to the highest level feature is generating by summing an RGB image corresponding to the highest level feature and an image obtained by upscaling the initial RGB image corresponding to the highest level feature.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
generating the latent code by performing convolution and full connection for the highest level feature.

### Claim 8 (depends on 1)

A non-transitory, computer-readable storage medium storing instructions that, in response to being executed by a processor, cause the processor to perform the method of claim 1.

### Claim 9

An apparatus, the apparatus comprising:
a pyramid feature obtaining processing element configured to obtain a pyramid feature of an input face image;
a first style feature obtaining processing element configured to generate a first initial style feature corresponding to a highest level feature based on the highest level feature of the pyramid feature, and a latent code and noise of the input face image, and generate a first style feature corresponding to each level feature based on an integrated feature corresponding to each level feature of the pyramid feature, the noise, and the latent code; and
a face image obtaining processing element configured to generate a high-quality face image of the input face image based on the first initial style feature and the first style feature corresponding to each level feature of the pyramid feature,
wherein the integrated feature corresponding to each level feature is obtained based on each level feature, the first style feature corresponding to a higher level feature of each level feature, the latent code, and the noise, and
wherein an integrated feature corresponding to the highest level feature of the pyramid feature is generated based on the highest level feature, the latent code, the noise, and the first initial style feature.

### Claim 10 (depends on 9)

The apparatus of claim 9, further comprising:
a second style feature obtaining processing element configured to:
generate a second style feature corresponding to the higher level feature based on the first style feature corresponding to the higher level feature, the noise, and the latent code;
generate the integrated feature corresponding to each level feature based on the second style feature corresponding to the higher level feature, and each level feature;
generate a second initial style feature corresponding to the highest level feature based on the first initial style feature, the latent code, and the noise; and
generate the integrated feature corresponding to the highest level feature based on the second initial style feature and the highest level feature.

### Claim 11 (depends on 10)

The apparatus of claim 10, wherein the second style feature obtaining processing element is configured to:
generate a first fully connected feature of the second style feature corresponding to the higher level feature and a first fully connected feature of each level feature by performing a full connection for each of the second style feature corresponding to the higher level feature, and each level feature;
generate a first feature by performing first weighted summation of the first fully connected feature of the second style feature corresponding to the higher level feature and the first fully connected feature of each level feature;
generate a second feature by performing the full connection for the first feature;
generate the integrated feature corresponding to each level feature based on the second feature, generate a first fully connected feature of the second initial style feature and a first fully connected feature of the highest level feature by performing the full connection for each of the second initial style feature and the highest level feature;
generate a third feature by performing second weighted summation of the first fully connected feature of the second initial style feature and the first fully connected feature of the highest level feature;
generate a fourth feature by performing the full connection for the third feature; and
obtain the integrated feature corresponding to the highest level feature based on the fourth feature.

### Claim 12 (depends on 11)

The apparatus of claim 11, wherein a first weight of the first weighted summation for generating the first feature is obtained based on the second style feature corresponding to the higher level feature, and
wherein a second weight of the second weighted summation for obtaining the third feature is obtained based on the second initial style feature.

### Claim 13 (depends on 12)

The apparatus of claim 12, further comprising:
a weight obtaining processing element configured to:
generate the first weight by performing a softmax operation for the first fully connected feature corresponding to the second style feature corresponding to the higher level feature; and
generate the second weight by performing the softmax operation for a second fully connected feature corresponding to the second initial style feature.

### Claim 14 (depends on 9)

The apparatus of claim 9, wherein the face image obtaining processing element is configured to:
obtain a red, green, blue (RGB) image corresponding to each level feature based on the first style feature corresponding to each level feature of the pyramid feature;
generate an initial RGB image corresponding to the highest level feature based on the first initial style feature; and
generate a cumulative RGB image corresponding to each level feature by summing the RGB image corresponding to each level feature and an upscaled RGB image corresponding to the higher level feature of each level feature,
wherein a cumulative RGB image corresponding to a lowest level feature is a high-quality image,
wherein the upscaled RGB image corresponding to each level feature is generated by upscaling the cumulative RGB image corresponding to each level feature, and
wherein a cumulative RGB image corresponding to the highest level feature is generated by summing an RGB image corresponding to the highest level feature and an image generated by upscaling the initial RGB image corresponding to the highest level feature.

### Claim 15 (depends on 9)

The apparatus of claim 9, further comprising:
a latent code obtaining processing element configured to generate the latent code by performing convolution and full connection for the highest level feature.

### Claim 16

An electronic device, comprising:
one or more processors configured to execute instructions; and
a memory storing the instructions, wherein execution of the instructions configures the processors to:
obtain a pyramid feature of an input face image;
generate a first initial style feature corresponding to a highest level feature based on the highest level feature of the pyramid feature, and a latent code and noise of the input face image;
generate a first style feature corresponding to each level feature based on an integrated feature corresponding to each level feature of the pyramid feature, the noise, and the latent code; and
generate a high-quality face image of the input face image based on the first initial style feature and the first style feature corresponding to each level feature of the pyramid feature,
wherein the integrated feature corresponding to each level feature is generated based on each level feature, the first style feature corresponding to a higher level feature of each level feature, the latent code, and the noise, and
wherein an integrated feature corresponding to the highest level feature of the pyramid feature is generated based on the highest level feature, the latent code, the noise, and the first initial style feature.
