# US 2024/0005466 A1 · Animal face style image generation method and apparatus, model training method and apparatus, and device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0005466 A1
- Document ID: US20240005466A1
- Status: Published application
- Kind: A1
- Application number: 18/252,855
- Filed: 2021-11-12
- Published: 2024-01-04
- Earliest claimed: 2020-11-13
- Assignees: Beijing Zitiao Network Technology Co., Ltd.
- Applicants: Beijing Zitiao Network Technology Co., Ltd.
- Inventors: Qian HE
- Web page: https://patentgrep.com/patent/US20240005466A1

## CPC

- G06V 40/168: {Feature extraction; Face representation}
- G06N 3/094: Adversarial learning
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 7/12: Edge-based segmentation
- G06T 11/00: Two-dimensional [2D] image generation
- G06T 11/60: Creating or editing images; Combining images with text
- G06V 10/803: {of input or preprocessed data}
- G06V 10/806: {of extracted features}
- G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships}
- G06N 3/045: Combinations of networks
- G06N 3/0475: Generative networks
- G06T 2207/20081: Training; Learning
- G06T 2207/20221: Image fusion; Image merging
- G06T 2207/30201: Face
- Y02A 40/70: in livestock or poultry
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- Y02A: TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE

## Abstract

An animal face style image generation method, a model training method, and a device are provided. The method includes: acquiring an original human face image; and obtaining an animal face style image corresponding to the original human face image. The animal face style image refers to an image obtained by transforming a human face on the original human face image into an animal face, the animal face style image generation model is obtained by training with a first human face sample image and a first animal face style sample image, the first animal face style sample image is generated from the first human face sample image by a pre-trained animal face generation model, and the animal face generation model is obtained by training with a second human face sample image and a first animal face sample image.

## Claims

### Claim 1

A method for generating an animal face style image, comprising:
acquiring an original human face image; and
obtaining an animal face style image corresponding to the original human face image by using a pre-trained animal face style image generation model;
wherein the animal face style image refers to an image obtained by transforming a human face on the original human face image into an animal face, the animal face style image generation model is obtained by training with a first human face sample image and a first animal face style sample image, the first animal face style sample image is generated from the first human face sample image by a pre-trained animal face generation model, and the animal face generation model is obtained by training with a second human face sample image and a first animal face sample image.

### Claim 2 (depends on 1)

The method according to claim 1, further comprising:
determining, based on an animal special effect type selected by a user, a correspondence between animal face key points and human face key points, wherein the correspondence corresponds to the animal special effect type; and
performing, based on the correspondence between the animal face key points and the human face key points, human face position adjustment on a user image to obtain the original human face image, wherein the correspondence corresponds to the animal special effect type, and the original human face image meets input requirements of the animal face style image generation model.

### Claim 3 (depends on 2)

The method according to claim 2, further comprising:
fusing an animal face region in the animal face style image with a background region in the user image to obtain a target animal face style image corresponding to the user image.

### Claim 4 (depends on 3)

The method according to claim 3, wherein the fusing the animal face region in the animal face style image with the background region in the user image to obtain the target animal face style image corresponding to the user image comprises:
obtaining, based on the animal face style image, an intermediate image with a same image size as the user image, wherein a position of an animal face region in the intermediate image is the same as a position of a human face region in the user image;
determining a first animal face mask image corresponding to the animal special effect type; and
fusing, based on the first animal face mask image, the user image with the intermediate image to obtain the target animal face style image corresponding to the user image, wherein the first animal face mask image is used for determining the animal face region in the intermediate image as an animal face region in the target animal face style image.

### Claim 5 (depends on 1)

The method according to claim 1, wherein:
the first human face sample image is obtained by performing, based on a first correspondence between face key points in a first original human face sample image and animal face key points in a first original animal face sample image, human face position adjustment on the first original human face sample image;
the second human face sample image is obtained by performing, based on a second correspondence between face key points in a second original human face sample image and animal face key points in the first original animal face sample image, human face position adjustment on the second original human face sample image; and
the first animal face sample image is obtained by performing, based on the first correspondence or the second correspondence, animal face position adjustment on the first original animal face sample image.

### Claim 6 (depends on 1)

The method according to claim 1, wherein:
the animal face style image generation model is obtained by training with the first human face sample image and a second animal face style sample image, and the second animal face style sample image is obtained by replacing a background region in the first animal face style sample image with a background region in the first human face sample image.

### Claim 7 (depends on 6)

The method according to claim 6, wherein:
the second animal face style sample image is obtained by fusing, based on a second animal face mask image, the first animal face style sample image with the first human face sample image; and
the second animal face mask image is obtained from the first animal face style sample image by a pre-trained animal face segmentation model, wherein the second animal face mask image is used for determining an animal face region in the first animal face style sample image as an animal face region in the second animal face style sample image.

### Claim 8

A method for training an animal face style image generation model, comprising:
training an image generation model with a second human face sample image and a first animal face sample image to obtain an animal face generation model;
obtaining a first animal face style sample image corresponding to a first human face sample image by using the animal face generation model, wherein the first animal face style sample image refers to an image obtained by transforming a human face on the first human face sample image into an animal face; and
training a style image generation model with the first human face sample image and the first animal face style sample image to obtain an animal face style image generation model;
wherein, the animal face style image generation model is used for obtaining an animal face style image corresponding to an original human face image, and the animal face style image refers to an image obtained by transforming a human face on the original human face image into an animal face.

### Claim 9 (depends on 8)

The method according to claim 8, further comprising:
determining a second correspondence between human face key points in a second original human face sample image and animal face key points in a first original animal face sample image;
performing, based on the second correspondence, human face position adjustment on the second original human face sample image to obtain the second human face sample image; and
performing, based on the second correspondence, animal face position adjustment on the first original animal face sample image based on the second correspondence to obtain the first animal face sample image.

### Claim 10 (depends on 9)

The method according to claim 9, further comprising:
determining a first correspondence between face key points in a first original human face sample image and animal face key points in the first original animal face sample image; and
performing, based on the first correspondence, animal face position adjustment on the first original human face sample image to obtain the first human face sample image.

### Claim 11 (depends on 8)

The method according to claim 8, further comprising:
replacing a background region in the first animal face style sample image with a background region in the first human face sample image to obtain a second animal face style sample image; and
the training the style image generation model with the first human face sample image and the first animal face style sample image to obtain the animal face style image generation model comprises:
training the style image generation model with the first human face sample image and a second animal face style sample image to obtain the animal face style image generation model.

### Claim 12 (depends on 11)

The method according to claim 11, wherein the replacing the background region in the first animal face style sample image with the background region in the first human face sample image to obtain the second animal face style sample image comprises:
obtaining an animal face mask image corresponding to the first animal face style sample image by using a pre-trained animal face segmentation model; and
fusing, based on the animal face mask image, the first animal face style sample image with the first human face sample image to obtain the second animal face style sample image, wherein the animal face mask image is used for determining an animal face region in the first animal face style sample image as an animal face region in the second animal face style sample image.

### Claim 13 (depends on 12)

The method according to claim 12, further comprising:
acquiring a second animal face sample image and a marking of position of the animal face region in the second animal face sample image; and
obtaining the animal face segmentation model by training with the second animal face sample image and the marking of position of the animal face region.

### Claim at position 14

14-15. (canceled)

### Claim 16

An electronic device, comprising:
a processor; and
a memory storing a computer program that, when executed by the processor, causes the processor to implement, acquiring an original luman face image; and
obtaining an animal face style image corresponding to the original human face image by using a pre-trained animal face style image generation model,
wherein the animal face style image refers to an image obtained by transforming a human face on the original human face image into an animal face, the animal face style image generation model is obtained by training with a first human face sample image and a first animal face style sample image, the first animal face style sample image is generated from the first human face sample image by a pre-trained animal face generation model, and the animal face generation model is obtained by training with a second human face sample image and a first animal face sample image.

### Claim 17

(canceled)

### Claim 18

An electronic device, comprising:
a processor; and
a memory storing a computer program that, when executed by the processor, causes the processor to implement:
training an image generation model with a second human face sample image and a first animal face sample image to obtain an animal face generation model;
obtaining a first animal face style sample image corresponding to a first human face sample image by using the animal face generation model, wherein the first animal face style sample image refers to an image obtained by transforming a human face on the first human face sample image into an animal face; and
training a style image generation model with the first human face sample image and the first animal face style sample image to obtain an animal face style image generation model;
wherein, the animal face style image generation model is used for obtaining an animal face style image corresponding to an original human face image, and the animal face style image refers to an image obtained by transforming a human face on the original human face image into an animal face.

### Claim 19 (depends on 16)

The electronic device according to claim 16, wherein the processor further implements:
determining, based on an animal special effect type selected by a user, a correspondence between animal face key points and human face key points, wherein the correspondence corresponds to the animal special effect type; and
performing, based on the correspondence between the animal face key points and the human face key points, human face position adjustment on a user image to obtain the original human face image, wherein the correspondence corresponds to the animal special effect type, and the original human face image meets input requirements of the animal face style image generation model.

### Claim 20 (depends on 19)

The electronic device according to claim 19, wherein the processor further implements:
fusing an animal face region in the animal face style image with a background region in the user image to obtain a target animal face style image corresponding to the user image.

### Claim 21 (depends on 18)

The electronic device according to claim 18, wherein the processor further implements:
determining a second correspondence between human face key points in a second original human face sample image and animal face key points in a first original animal face sample image;
performing, based on the second correspondence, human face position adjustment on the second original human face sample image to obtain the second human face sample image; and
performing, based on the second correspondence, animal face position adjustment on the first original animal face sample image based on the second correspondence to obtain the first animal face sample image.

### Claim 22 (depends on 21)

The electronic device according to claim 21, wherein the processor further implements:
determining a first correspondence between face key points in a first original human face sample image and animal face key points in the first original animal face sample image; and
performing, based on the first correspondence, animal face position adjustment on the first original human face sample image to obtain the first human face sample image.
