# US 2026/0030725 A1 · Generating images using a machine learning model

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0030725 A1
- Document ID: US20260030725A1
- Status: Published application
- Kind: A1
- Application number: 18/788,079
- Filed: 2024-07-29
- Published: 2026-01-29
- Earliest claimed: 2024-07-29
- Assignees: Lemon Inc.
- Applicants: Lemon Inc.
- Inventors: Guoxian Song; You Xie; Hongyi Xu; Chao Wang; Linjie Luo
- Web page: https://patentgrep.com/patent/US20260030725A1

## CPC

- G06T 5/60: using machine learning, e.g. neural networks
- G06T 3/18: Image warping, e.g. rearranging pixels individually
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

The present disclosure describes techniques for generating images using a machine learning model. Features are extracted from a source image by a machine learning model. The source image comprises a portrait of a subject. A warp grid is generated based on the source image and a driving image by the machine learning model. The driving image depicts a pose or a visage. The warp grid indicates differences between the source image and the driving image. A warped source image is generated by applying the warp grid to the source image. A mask and a decoded image are generated based on the warp grid and the extracted features. An output image is generated based on the warped source image, the mask, and the decoded image. The output image depicts the subject having the pose or the visage.

## Claims

### Claim 1

A method of generating images using a machine learning model, comprising:
extracting features from a source image by an encoder of the machine learning model, wherein the source image comprises a portrait of a subject;
generating a warp grid based on the source image and a driving image by a motion estimator of the machine learning model, wherein the driving image depicts a pose or a visage, and wherein the warp grid indicates differences between the source image and the driving image;
generating a warped source image by applying the warp grid to the source image;
generating a mask and a decoded image by a decoder of the machine learning model based on the warp grid and the features extracted from the source image, wherein the mask indicates one or more regions in which original information from the source image is to be preserved; and
generating an output image based on the warped source image, the mask, and the decoded image, wherein the output image depicts the subject having the pose or the visage.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
replacing the driving image with a modified driving image to mitigate appearance leakage from the driving image, wherein the modified driving image depicts a different subject having the pose or the visage.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
utilizing the machine learning model to generate training pairs, wherein each training pair comprises the source image, the modified driving image, and the output image, and wherein the training pairs are utilized to train another machine learning model for generating portrait animations.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
applying a global loss based on comparing an entirety of the output image with an entirety of the driving image.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
applying a local region loss based on comparing local patches of the output image with corresponding local patches of the driving image.

### Claim 6 (depends on 5)

The method of claim 5, wherein the local patches comprise a local patch associated with a mouth region and local patches associated with eye regions.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
extracting the source image and the driving image from a same video.

### Claim 8 (depends on 1)

The method of claim 1, wherein the pose comprises a head pose, and the visage comprises a facial visage.

### Claim 9

A system of generating images using a machine learning model, comprising:
at least one processor; and
at least one memory communicatively coupled to the at least one processor and comprising computer-readable instructions that upon execution by the at least one processor cause the at least one processor to perform operations comprising:
extracting features from a source image by an encoder of the machine learning model, wherein the source image comprises a portrait of a subject;
generating a warp grid based on the source image and a driving image by a motion estimator of the machine learning model, wherein the driving image depicts a pose or a visage, and wherein the warp grid indicates differences between the source image and the driving image;
generating a warped source image by applying the warp grid to the source image;
generating a mask and a decoded image by a decoder of the machine learning model based on the warp grid and the features extracted from the source image, wherein the mask indicates one or more regions in which original information from the source image is to be preserved; and
generating an output image based on the warped source image, the mask, and the decoded image, wherein the output image depicts the subject having the pose or the visage.

### Claim 10 (depends on 9)

The system of claim 9, the operations further comprising:
replacing the driving image with a modified driving image to mitigate appearance leakage from the driving image, wherein the modified driving image depicts a different subject having the pose or the visage.

### Claim 11 (depends on 9)

The system of claim 9, the operations further comprising:
utilizing the machine learning model to generate training pairs, wherein each training pair comprises the source image, the modified driving image, and the output image, and wherein the training pairs are utilized to train another machine learning model for generating portrait animations.

### Claim 12 (depends on 9)

The system of claim 9, the operations further comprising:
applying a global loss based on comparing an entirety of the output image with an entirety of the driving image.

### Claim 13 (depends on 9)

The system of claim 9, the operations further comprising:
applying a local region loss based on comparing local patches of the output image with corresponding local patches of the driving image.

### Claim 14 (depends on 13)

The system of claim 13, wherein the local patches comprise a local patch associated with a mouth region and local patches associated with eye regions.

### Claim 15 (depends on 9)

The system of claim 9, wherein the pose comprises a head pose, and the visage comprises a facial visage.

### Claim 16

A non-transitory computer-readable storage medium, storing computer-readable instructions that upon execution by a processor cause the processor to implement operations comprising:
extracting features from a source image by an encoder of the machine learning model, wherein the source image comprises a portrait of a subject;
generating a warp grid based on the source image and a driving image by a motion estimator of the machine learning model, wherein the driving image depicts a pose or a visage, and wherein the warp grid indicates differences between the source image and the driving image;
generating a warped source image by applying the warp grid to the source image;
generating a mask and a decoded image by a decoder of the machine learning model based on the warp grid and the features extracted from the source image, wherein the mask indicates one or more regions in which original information from the source image is to be preserved; and
generating an output image based on the warped source image, the mask, and the decoded image, wherein the output image depicts the subject having the pose or the visage.

### Claim 17 (depends on 16)

The non-transitory computer-readable storage medium of claim 16, the operations further comprising:
replacing the driving image with a modified driving image to mitigate appearance leakage from the driving image, wherein the modified driving image depicts a different subject having the pose or the visage.

### Claim 18 (depends on 16)

The non-transitory computer-readable storage medium of claim 16, the operations further comprising:
utilizing the machine learning model to generate training pairs, wherein each training pair comprises the source image, the modified driving image, and the output image, and wherein the training pairs are utilized to train another machine learning model for generating portrait animations.

### Claim 19 (depends on 16)

The non-transitory computer-readable storage medium of claim 16, the operations further comprising:
applying a global loss based on comparing an entirety of the output image with an entirety of the driving image.

### Claim 20 (depends on 16)

The non-transitory computer-readable storage medium of claim 16, the operations further comprising:
applying a local region loss based on comparing local patches of the output image with corresponding local patches of the driving image, wherein the local patches comprise a local patch associated with a mouth region and local patches associated with eye regions.
