# US 2023/0054283 A1 · Methods and apparatuses for generating style pictures

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0054283 A1
- Document ID: US20230054283A1
- Status: Published application
- Kind: A1
- Application number: 17/408,107
- Filed: 2021-08-20
- Published: 2023-02-23
- Earliest claimed: 2021-08-20
- Assignees: Kwai Inc.
- Applicants: Kwai Inc.
- Inventors: Jiayi LIU; Shen WANG; Zhenyu LIAO; Huayan WANG
- Web page: https://patentgrep.com/patent/US20230054283A1

## CPC

- G06T 11/60: Creating or editing images; Combining images with text
- G06F 18/21: Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06N 3/0475: Generative networks
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06N 3/096: Transfer learning
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 11/00: Two-dimensional [2D] image generation
- G06V 10/82: using neural networks
- G06V 40/161: {Detection; Localisation; Normalisation}
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/20221: Image fusion; Image merging
- G06T 2207/30201: Face
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A style picture generating method, an apparatus and a non-transitory computer readable storage medium thereof are provided. The method includes: obtaining one or more models by training a neural network; obtaining a plurality of interpolated models based on the one or more models; generating a plurality of pictures by the plurality of interpolated models; and generating the style picture by combining two or more pictures in the plurality of pictures using one or more model-specific alpha masks.

## Claims

### Claim 1

A method for generating a style picture, comprising:
obtaining one or more models by training a neural network;
obtaining a plurality of interpolated models based on the one or more models;
generating a plurality of pictures by the plurality of interpolated models; and
generating the style picture by combining two or more pictures in the plurality of pictures using one or more model-specific alpha masks.

### Claim 2 (depends on 1)

The method of claim 1, wherein the plurality of interpolated models comprise a first interpolated model and a second interpolated model; and
wherein the method further comprises:
generating a first picture by the first interpolated model and generating a second picture by the second interpolated model; and
generating the style picture by combining the first picture and the second picture using a first model-specific alpha mask in the one or more model-specific alpha masks.

### Claim 3 (depends on 2)

The method of claim 2, wherein generating the style picture by combining the first picture and the second picture using the first model-specific alpha mask comprises:
identifying a facial target area in the first picture and determining an intermediate matrix for the facial target area;
obtaining an alpha mask matrix by performing convolution operations on the intermediate matrix; and
generating the style picture based on the alpha mask matrix, the first picture, and the second picture.

### Claim 4 (depends on 3)

The method of claim 3, wherein each picture of a plurality of pictures generated by the first interpolated model comprises the facial target area.

### Claim 5 (depends on 3)

The method of claim 3, wherein performing the convolution operations on the intermediate matrix comprises:
performing the convolution operations on the intermediate matrix by using a kernel function.

### Claim 6 (depends on 1)

The method of claim 1, wherein the plurality of interpolated models comprise a first interpolated model, a second interpolated model, and a third interpolated model; and
wherein the method further comprises:
respectively generating a first picture, a second picture, and a third picture by the first interpolated model, the second interpolated model, and the third interpolated model;
generating a first combined picture by combining the first picture and the second picture using a first model-specific alpha mask in the one or more model-specific alpha masks; and
generating the style picture by combining the first combined picture and the third picture using a second model-specific alpha mask in the one or more model-specific alpha masks.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
obtaining a base model by training the neural network using a face dataset;
generating one or more transferred models by training the base model using one or more new datasets; and
obtaining the plurality of interpolated models based on at least one the base model or the one or more transferred models.

### Claim 8 (depends on 7)

The method of claim 7, wherein generating the one or more transferred models by training the base model using the one or more new datasets comprises:
generating a plurality of different transferred models in different training periods by training the base model on a dataset of a style; or
generating the plurality of different transferred models by training the base model on a plurality of datasets of different styles.

### Claim 9 (depends on 1)

The method of claim 1, wherein obtaining the plurality of interpolated models based on the one or more models comprises:
obtaining the plurality of interpolated models by interpolating at different layers in one of the one or more models.

### Claim 10 (depends on 1)

The method of claim 1, wherein the neural network is a style-based generative adversarial network (GAN).

### Claim 11

An apparatus for generating a style picture, comprising:
one or more processors; and
a memory configured to store instructions executable by the one or more processors; wherein the one or more processors, upon execution of the instructions, are configured to:
obtain one or more models by training a neural network;
obtain a plurality of interpolated models based on the one or more models;
generate a plurality of pictures by the plurality of interpolated models; and
generate the style picture by combining two or more pictures in the plurality of pictures using one or more model-specific alpha masks.

### Claim 12 (depends on 11)

The apparatus of claim 11, wherein the plurality of interpolated models comprise a first interpolated model and a second interpolated model; and
wherein the one or more processors are further configured to:
generate a first picture by the first interpolated model and generating a second picture by the second interpolated model; and
generate the style picture by combining the first picture and the second picture using a first model-specific alpha mask in the one or more model-specific alpha masks.

### Claim 13 (depends on 12)

The apparatus of claim 12, wherein the one or more processors are further configured to generate the style picture by combining the first picture and the second picture using the first model-specific alpha mask comprises that the one or more processors are further configured to:
identify a facial target area in the first picture and determining an intermediate matrix for the facial target area;
obtain an alpha mask matrix by performing convolution operations on the intermediate matrix; and
generate the style picture based on the alpha mask matrix, the first picture, and the second picture.

### Claim 14 (depends on 13)

The apparatus of claim 13, wherein each picture of a plurality of pictures generated by the first interpolated model comprises the facial target area.

### Claim 15 (depends on 11)

The apparatus of claim 11, wherein the plurality of interpolated models comprise a first interpolated model, a second interpolated model, and a third interpolated model; and
wherein the one or more processors are further configured to:
respectively generate a first picture, a second picture, and a third picture by the first interpolated model, the second interpolated model, and the third interpolated model;
generate a first combined picture by combining the first picture and the second picture using a first model-specific alpha mask in the one or more model-specific alpha masks; and
generate the style picture by combining the first combined picture and the third picture using a second model-specific alpha mask in the one or more model-specific alpha masks.

### Claim 16 (depends on 11)

The apparatus of claim 11, wherein the one or more processors are further configured to:
obtaining a base model by training the neural network using a face dataset;
generating one or more transferred models by training the base model using one or more new datasets; and
obtaining the plurality of interpolated models based on at least one of the base model or the one or more transferred models.

### Claim 17

A non-transitory computer readable storage medium, comprising instructions stored therein, wherein, upon execution of the instructions by one or more processors, the instructions cause the one or more processors to perform acts comprising:
obtaining one or more models by training a neural network;
obtaining a plurality of interpolated models based on the one or more models;
generating a plurality of pictures by the plurality of interpolated models; and
generating the style picture by combining two or more pictures in the plurality of pictures using one or more model-specific alpha masks.

### Claim 18 (depends on 17)

The non-transitory computer readable storage medium of claim 17, wherein the plurality of interpolated models comprise a first interpolated model and a second interpolated model; and
wherein the instructions cause the one or more processors to perform acts further comprising:
generating a first picture by the first interpolated model and generating a second picture by the second interpolated model; and
generating the style picture by combining the first picture and the second picture using a first model-specific alpha mask in the one or more model-specific alpha masks.

### Claim 19 (depends on 18)

The non-transitory computer readable storage medium of claim 18, wherein generating the style picture by combining the first picture and the second picture using the first model-specific alpha mask comprises:
identifying a facial target area in the first picture and determining an intermediate matrix for the facial target area;
obtaining an alpha mask matrix by performing convolution operations on the intermediate matrix; and
generating the style picture based on the alpha mask matrix, the first picture, and the second picture.

### Claim 20 (depends on 19)

The non-transitory computer readable storage medium of claim 19, wherein each picture of a plurality of pictures generated by the first interpolated model comprises the facial target area.
