# US 2025/0054105 A1 · Image processing method, apparatus, and device, and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0054105 A1
- Document ID: US20250054105A1
- Status: Published application
- Kind: A1
- Application number: 18/717,682
- Filed: 2022-12-02
- Published: 2025-02-13
- Earliest claimed: 2021-12-10
- Assignees: Beijing Zitiao Network Technology Co., Ltd.
- Applicants: Beijing Zitiao Network Technology Co., Ltd.
- Inventors: Panpan XU; Miao HUA
- Web page: https://patentgrep.com/patent/US20250054105A1

## CPC

- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06F 18/00: Pattern recognition
- G06F 18/253: {of extracted features}
- G06V 10/806: {of extracted features}
- G06V 10/82: using neural networks
- G06V 40/168: {Feature extraction; Face representation}
- G06T 2207/20221: Image fusion; Image merging
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Disclosed in examples of the disclosure are a method, apparatus, device, and storage medium for processing image. The method includes: extracting facial feature information of a first character appearance image and structural feature information of a second character appearance image (S110); inputting the facial feature information and the structural feature information into a first character appearance fusion model, and obtaining a third character appearance image (S120); training a second character appearance fusion model based on the first character appearance image, the second character appearance image and the third character appearance image (S130); and inputting a source character appearance image and a target character appearance image into the trained second character appearance fusion model to obtain a new character appearance image (S140).

## Claims

### Claim 1

A method for processing an image, comprising:
extracting facial feature information of a first character appearance image and structural feature information of a second character appearance image;
inputting the facial feature information and the structural feature information into a first character appearance fusion model to obtain a third character appearance image;
training a second character appearance fusion model based on the first character appearance image, the second character appearance image and the third character appearance image; and
inputting a source character appearance image and a target character appearance image into the trained second character appearance fusion model to obtain a new character appearance image.

### Claim 2 (depends on 1)

The method of claim 1, wherein the first character appearance fusion model includes a feature injection module and a first decoder, and inputting the facial feature information and the structural feature information into a first character appearance fusion model to obtain a third character appearance image comprises:
inputting the facial feature information and the structural feature information into the feature injection module to obtain fused feature information; and
inputting the fused feature information into the first decoder to output the third character appearance image.

### Claim 3 (depends on 2)

The method of claim 2, wherein the first character appearance fusion model further includes a first encoder, the first encoder is hop-connected with the first decoder; and inputting the facial feature information and the structural feature information into a first character appearance fusion model to obtaining a third character appearance image comprises:
inputting the second character appearance image into the first encoder to obtain first encoding information;
inputting the first encoding information, the facial feature information, and the structural feature information into the feature injection module to obtain fused feature information; and
inputting the fused feature information, the first encoding information, and the structural feature information into the first decoder to obtain the third character appearance image.

### Claim 4 (depends on 3)

The method of claim 3, wherein the feature injection module comprises a fully-connected layer, a convolutional layer and an injection layer; and inputting the first encoding information, the facial feature information, and the structural feature information into the feature injection module to obtain fused feature information comprises:
inputting the facial feature information into the fully-connected layer to obtain intermediate facial feature information;
inputting the structural feature information into the convolutional layer to obtain intermediate structural feature information; and
inputting the intermediate facial feature information, the intermediate structural feature information, and the first encoding information into the injection layer for fusion to output the fused feature information.

### Claim 5 (depends on 1)

The method of claim 1, wherein the second character appearance fusion model comprises a second encoder and a second decoder; and inputting a source character appearance image and a target character appearance image into the trained second character appearance fusion model to obtain a new character appearance image comprises:
inputting the source character appearance image and the target character appearance image into the second encoder to obtain second encoding information; and
inputting the second encoding information into the second decoder to obtain the new character appearance image.

### Claim 6 (depends on 1)

The method of claim 1, wherein training the first character appearance fusion model comprises:
extracting first facial feature sample information of a first character appearance sample and first structural feature sample information of a second character appearance sample;
inputting the first facial feature sample information and the first structural feature sample information into the first character appearance fusion model to output a third character appearance sample;
extracting second facial feature sample information and second structural feature sample information of the third character appearance sample;
determining a first loss function of the second facial feature sample information and the first facial feature sample information, and a second loss function of the second structural feature sample information and the first structural feature sample information; and
training the first character appearance fusion model based on the first loss function and the second loss function.

### Claim 7 (depends on 1)

The method of claim 1, wherein training a second character appearance fusion model based on the first character appearance image, the second character appearance image, and the third character appearance image comprises:
inputting the first character appearance image and the second character appearance image into the second character appearance fusion model to obtain a fused character appearance image;
determining a third loss function of the fused character appearance image and the third character appearance image; and
training the second character appearance fusion model based on the third loss function.

### Claim 8

(canceled)

### Claim 9

An electronic device, comprising:
at least one processor; and
a memory configured to store at least one program which, when executed by the at least one processor, causes the at least one processor to implement the method, comprising:
extracting facial feature information of a first character appearance image and structural feature information of a second character appearance image;
inputting the facial feature information and the structural feature information into a first character appearance fusion model to obtain a third character appearance image;
training a second character appearance fusion model based on the first character appearance image, the second character appearance image and the third character appearance image; and
inputting a source character appearance image and a target character appearance image into the trained second character appearance fusion model to obtain a new character appearance image.

### Claim 10

A computer-readable storage medium, storing a computer program which, when executed by a processor, causes the processor to implement the method, comprising:
extracting facial feature information of a first character appearance image and structural feature information of a second character appearance image;
inputting the facial feature information and the structural feature information into a first character appearance fusion model to obtain a third character appearance image;
training a second character appearance fusion model based on the first character appearance image, the second character appearance image and the third character appearance image; and
inputting a source character appearance image and a target character appearance image into the trained second character appearance fusion model to obtain a new character appearance image.

### Claim 11 (depends on 1)

The method of claim 1, including a computer program product which, when executed by a computer, causes the computer to implement the method, comprising:
extracting facial feature information of a first character appearance image and structural feature information of a second character appearance image;
inputting the facial feature information and the structural feature information into a first character appearance fusion model to obtain a third character appearance image;
training a second character appearance fusion model based on the first character appearance image, the second character appearance image and the third character appearance image; and
inputting a source character appearance image and a target character appearance image into the trained second character appearance fusion model to obtain a new character appearance image.

### Claim 12 (depends on 9)

The electronic device of claim 9, wherein the first character appearance fusion model includes a feature injection module and a first decoder, and inputting the facial feature information and the structural feature information into a first character appearance fusion model to obtain a third character appearance image comprises:
inputting the facial feature information and the structural feature information into the feature injection module to obtain fused feature information; and
inputting the fused feature information into the first decoder to output the third character appearance image.

### Claim 13 (depends on 12)

The electronic device of claim 12, wherein the first character appearance fusion model further includes a first encoder, the first encoder is hop-connected with the first decoder; and inputting the facial feature information and the structural feature information into a first character appearance fusion model to obtaining a third character appearance image comprises:
inputting the second character appearance image into the first encoder to obtain first encoding information;
inputting the first encoding information, the facial feature information, and the structural feature information into the feature injection module to obtain fused feature information; and
inputting the fused feature information, the first encoding information, and the structural feature information into the first decoder to obtain the third character appearance image.

### Claim 14 (depends on 13)

The electronic device of claim 13, wherein the feature injection module comprises a fully-connected layer, a convolutional layer and an injection layer; and inputting the first encoding information, the facial feature information, and the structural feature information into the feature injection module to obtain fused feature information comprises:
inputting the facial feature information into the fully-connected layer to obtain intermediate facial feature information;
inputting the structural feature information into the convolutional layer to obtain intermediate structural feature information; and
inputting the intermediate facial feature information, the intermediate structural feature information, and the first encoding information into the injection layer for fusion to output the fused feature information.

### Claim 15 (depends on 9)

The electronic device of claim 9, wherein the second character appearance fusion model comprises a second encoder and a second decoder; and inputting a source character appearance image and a target character appearance image into the trained second character appearance fusion model to obtain a new character appearance image comprises:
inputting the source character appearance image and the target character appearance image into the second encoder to obtain second encoding information; and
inputting the second encoding information into the second decoder to obtain the new character appearance image.

### Claim 16 (depends on 9)

The electronic device of claim 9, wherein training the first character appearance fusion model comprises:
extracting first facial feature sample information of a first character appearance sample and first structural feature sample information of a second character appearance sample;
inputting the first facial feature sample information and the first structural feature sample information into the first character appearance fusion model to output a third character appearance sample;
extracting second facial feature sample information and second structural feature sample information of the third character appearance sample;
determining a first loss function of the second facial feature sample information and the first facial feature sample information, and a second loss function of the second structural feature sample information and the first structural feature sample information; and
training the first character appearance fusion model based on the first loss function and the second loss function.

### Claim 17 (depends on 16)

The electronic device of claim 16, wherein training a second character appearance fusion model based on the first character appearance image, the second character appearance image, and the third character appearance image comprises:
inputting the first character appearance image and the second character appearance image into the second character appearance fusion model to obtain a fused character appearance image;
determining a third loss function of the fused character appearance image and the third character appearance image; and
training the second character appearance fusion model based on the third loss function.

### Claim 18 (depends on 10)

The computer-readable storage medium of claim 10, wherein the first character appearance fusion model includes a feature injection module and a first decoder, and inputting the facial feature information and the structural feature information into a first character appearance fusion model to obtain a third character appearance image comprises:
inputting the facial feature information and the structural feature information into the feature injection module to obtain fused feature information; and
inputting the fused feature information into the first decoder to output the third character appearance image.

### Claim 19 (depends on 18)

The computer-readable storage medium of claim 18, wherein the first character appearance fusion model further includes a first encoder, the first encoder is hop-connected with the first decoder; and inputting the facial feature information and the structural feature information into a first character appearance fusion model to obtaining a third character appearance image comprises:
inputting the second character appearance image into the first encoder to obtain first encoding information;
inputting the first encoding information, the facial feature information, and the structural feature information into the feature injection module to obtain fused feature information; and
inputting the fused feature information, the first encoding information, and the structural feature information into the first decoder to obtain the third character appearance image.

### Claim 20 (depends on 19)

The computer-readable storage medium of claim 19, wherein the feature injection module comprises a fully-connected layer, a convolutional layer and an injection layer;
and inputting the first encoding information, the facial feature information, and the structural feature information into the feature injection module to obtain fused feature information comprises:
inputting the facial feature information into the fully-connected layer to obtain intermediate facial feature information;
inputting the structural feature information into the convolutional layer to obtain intermediate structural feature information; and
inputting the intermediate facial feature information, the intermediate structural feature information, and the first encoding information into the injection layer for fusion to output the fused feature information.

### Claim 21 (depends on 10)

The computer-readable storage medium of claim 10, wherein the second character appearance fusion model comprises a second encoder and a second decoder; and inputting a source character appearance image and a target character appearance image into the trained second character appearance fusion model to obtain a new character appearance image comprises:
inputting the source character appearance image and the target character appearance image into the second encoder to obtain second encoding information; and
inputting the second encoding information into the second decoder to obtain the new character appearance image.
