# US 2025/0097545 A1 · Video generation method, apparatus, device and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0097545 A1
- Document ID: US20250097545A1
- Status: Published application
- Kind: A1
- Application number: 18/711,530
- Filed: 2022-11-18
- Published: 2025-03-20
- Earliest claimed: 2021-11-18
- Assignees: Lemon Inc.
- Applicants: Lemon Inc.
- Inventors: Weibo GONG; Xiaojie JIN; Ding LIU; Xiaohui SHEN
- Web page: https://patentgrep.com/patent/US20250097545A1

## CPC

- H04N 21/816: {involving special video data, e.g 3D video}
- G06T 13/00: Animation
- G06V 10/42: Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
- G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V 10/75: Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- H04N 21/8113: {comprising music, e.g. song in MP3 format}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

The embodiments of the present disclosure provide a video generation method, an apparatus, a device, and a storage medium, the video generation method including: obtaining a plurality of images and music matched to the plurality of images; determining first feature information for the plurality of images and second feature information for the music; according to the first feature information, the second feature information and a plurality of pre-stored rendering effects, determining a target rendering effect combination; the rendering effects being animation, special effects or a transition; and generating a video according to the plurality of images, the music and the target rendering effect combination.

## Claims

### Claim 1

A video generation method, comprising:
acquiring a plurality of images and music matched with the plurality of images;
determining first feature information of the plurality of images and second feature information of the music;
determining a target rendering effect combination according to the first feature information, the second feature information and a pre-stored plurality of rendering effects, wherein the rendering effects are animation, special effect or transition; and
generating a video according to the plurality of images, the music and the target rendering effect combination.

### Claim 2 (depends on 1)

The method according to claim 1, wherein:
the first feature information comprises a first global feature and a first local feature;
the second feature information comprises a second global feature and a second local feature; and
the determining a target rendering effect combination according to the first feature information, the second feature information and a pre-stored plurality of rendering effects, comprises:
determining a plurality of candidate effects in the plurality of rendering effects according to the first global feature and the second global feature;
determining one or more target effects in the plurality of candidate effects according to the first local feature, and performing combination processing on the one or more target effects to obtain one or more rendering combinations; and
determining the target rendering effect combination according to the first local feature, the second local feature and the one or more rendering combinations.

### Claim 3 (depends on 2)

The method according to claim 2, wherein:
the first global feature comprises a first image emotion, a first image style and a first image scene corresponding to the plurality of images;
the second global feature comprises a first music emotion, a first music style and a first music theme; and
the determining a plurality of candidate effects in the plurality of rendering effects according to the first global feature and the second global feature, comprises:
for each rendering effect, acquiring a first initial score corresponding to each of the first image emotion, the first image style and the first image scene according to an identification of the each rendering effect;
screening the plurality of rendering effects according to the first initial score to obtain a plurality of intermediate effects;
for each intermediate effect, acquiring a second initial score corresponding to each of the first music emotion, the first music style and the first music theme according to an identification of the each intermediate effect; and
screening the plurality of intermediate effects according to the second initial score to obtain the plurality of candidate effects.

### Claim 4 (depends on 3)

The method according to claim 3, wherein the screening the plurality of rendering effects according to the first initial score to obtain a plurality of intermediate effect, comprises:
for each rendering effect, determining a sum of the first initial score corresponding to each of the first image emotion, the first image style and the first image scene, as a first target score corresponding to the each rendering effect; and
determining rendering effects of which the first target score is greater than or equal to a first threshold in the plurality of rendering effects, as the plurality of intermediate effects.

### Claim 5 (depends on 3)

The method according to claim 3, wherein the screening the plurality of intermediate effects according to the second initial score to obtain the plurality of candidate effects, comprises:
for each intermediate effect, determining a sum of the second initial score corresponding to each of the first music emotion, the first music style and the first music theme, as a second target score corresponding to the each intermediate effect; and
determining intermediate effects of which the second target score is greater than or equal to a second threshold in the plurality of intermediate effects, as the plurality of candidate effects.

### Claim 6 (depends on 2)

The method according to claim 2, wherein:
the first local feature comprises a second image emotion, a second image style and a second image scene corresponding to each of the plurality of images; and
the determining one or more target effects in the plurality of candidate effects according to the first local feature, comprises:
for each image, determining a third target score corresponding to each of the plurality of candidate effects under a condition of the each image, according to the plurality of candidate effects and the second image emotion, the second image style and the second image scene corresponding to the each image; and
determining the one or more target effects in the plurality of candidate effects according to the third target score corresponding to each of the plurality of candidate effects.

### Claim 7 (depends on 2)

The method according to claim 2, wherein:
the first local feature comprises a second image emotion, a second image style and a second image scene corresponding to each of the plurality of images;
the second local feature comprises a chorus point, a phrase and section point and a beat point of a music fragment corresponding to each of the plurality of images in the music; and
the determining the target rendering effect combination according to the first local feature, the second local feature and the one or more rendering combinations, comprises:
screening the one or more rendering combinations according to a second image emotion, a second image style and a second image scene corresponding to a first image in the plurality of images, and a chorus point, a phrase and section point and a beat point of a music fragment corresponding to the first image, to obtain N initial candidate combinations, where N is an integer greater than or equal to 1;
determining M (j−1)th candidate combinations according to the N initial candidate combinations and the one or more rendering combinations, where M is equal to a product of N and a total number of the one or more rendering combinations;
screening the M (j−1)th candidate combinations according to a second image emotion, a second image style and a second image scene corresponding to a jth image, and a chorus point, a phrase and section point and a beat point of a music fragment corresponding to the jth image, to determine N jth candidate combinations, and taking the N jth candidate combinations as new N initial candidate combinations, adding 1 to j, and repeating the steps until a last image in the plurality of images, where an initial value of j is 2; and
determining a candidate combination corresponding to the last image as the target rendering effect combination.

### Claim 8 (depends on 7)

The method according to claim 7, wherein the screening the one or more rendering combinations according to a second image emotion, a second image style and a second image scene corresponding to a first image in the plurality of images, and a chorus point, a phrase and section point and a beat point of a music fragment corresponding to the first image, to obtain N initial candidate combinations, comprises:
determining a combination score corresponding to each of the one or more rendering combinations, according to the second image emotion, the second image style and the second image scene corresponding to the first image, and the chorus point, the phrase and section point and the beat point of the music fragment corresponding to the first image; and
determining N rendering combinations of which the combination score is greater than or equal to a fourth threshold in the one or more rendering combinations, as the N initial candidate combinations.

### Claim 9 (depends on 8)

The method according to claim 8, wherein the determining a combination score corresponding to each of the one or more rendering combinations, according to the second image emotion, the second image style and the second image scene corresponding to the first image, and the chorus point, the phrase and section point and the beat point of the music fragment corresponding to the first image, comprises:
for each rendering combination in the one or more rendering combinations, determining a music matching score according to the chorus point, the phrase and section point and the beat point of the music fragment corresponding to the first image and an identification of each rendering effect in the each rendering combination;
determining an image matching score according to the second image emotion, the second image style and the second image scene corresponding to the first image and the identification of each rendering effect in the each rendering combination;
determining an internal combination score corresponding to the first image; and
determining the music matching score, the image matching score and the internal combination score as the combination score corresponding to the each rendering combination.

### Claim 10 (depends on 1)

The method according to claim 1, wherein the determining first feature information of the plurality of images and second feature information of the music, comprises:
performing feature extraction on the plurality of images through a pre-stored image feature extraction model, to obtain the first feature information of the plurality of images; and
performing feature extraction on the music through a pre-stored music feature extraction model, to obtain the second feature information.

### Claim 11 (depends on 1)

The method according to claim 1, wherein:
the target rendering effect combination comprises the animation, special effect and transition corresponding to each of the plurality of images;
the generating a video according to the plurality of images, the music and the target rendering effect combination, comprises:
sequentially displaying the plurality of images according to the animation, special effect and transition corresponding to each of the plurality of images in the target rendering effect combination, and playing the music, to generate the video.

### Claim 12 (depends on 1)

The method according to claim 1, wherein the acquiring a plurality of images and music matched with the plurality of images, comprises:
in response to a selection operation on a plurality of target images in a plurality of candidate images, determining the plurality of target images as the plurality of images; and
in response to a selection operation on target music in a plurality of candidate music, determining the target music as the music matched with the plurality of images.

### Claim 13

A video generation apparatus, comprising:
an acquisition module configured to acquire a plurality of images and music matched with the plurality of images;
a first determination module configured to determine first feature information of the plurality of images and second feature information of the music;
a second determination module configured to determine a target rendering effect combination according to the first feature information, the second feature information and a pre-stored plurality of rendering effects; the rendering effects being animation, special effect or transition; and
a generation module configured to generate a video according to the plurality of images, the music and the target rendering effect combination.

### Claim 14 (depends on 1)

An electronic device, comprising: a processor, and a memory communicatively connected to the processor;
the memory storing computer-executable instructions;
the processor executing the computer-executable instructions stored in the memory to implement the method according to claim 1.

### Claim 15

A non-transitory computer-readable storage medium, having computer-executable instructions stored thereon, which, in response to being executed by a processor, implement the method, comprising:
acquiring a plurality of images and music matched with the plurality of images;
determining first feature information of the plurality of images and second feature information of the music;
determining a target rendering effect combination according to the first feature information, the second feature information and a pre-stored plurality of rendering effects, wherein the rendering effects are animation, special effect or transition; and
generating a video according to the plurality of images, the music and the target rendering effect combination.

### Claim 16 (depends on 1)

A computer program product comprising a computer program which, in response to being executed by a processor, implements the method according to claim 1.

### Claim 17

(canceled)

### Claim 18 (depends on 15)

The non-transitory computer-readable storage medium according to claim 15, wherein:
the first feature information comprises a first global feature and a first local feature;
the second feature information comprises a second global feature and a second local feature; and
the determining a target rendering effect combination according to the first feature information, the second feature information and a pre-stored plurality of rendering effects, comprises:
determining a plurality of candidate effects in the plurality of rendering effects according to the first global feature and the second global feature;
determining one or more target effects in the plurality of candidate effects according to the first local feature, and performing combination processing on the one or more target effects to obtain one or more rendering combinations; and
determining the target rendering effect combination according to the first local feature, the second local feature and the one or more rendering combinations.

### Claim 19 (depends on 18)

The non-transitory computer-readable storage medium according to claim 18, wherein:
the first global feature comprises a first image emotion, a first image style and a first image scene corresponding to the plurality of images;
the second global feature comprises a first music emotion, a first music style and a first music theme; and
the determining a plurality of candidate effects in the plurality of rendering effects according to the first global feature and the second global feature, comprises:
for each rendering effect, acquiring a first initial score corresponding to each of the first image emotion, the first image style and the first image scene according to an identification of the each rendering effect;
screening the plurality of rendering effects according to the first initial score to obtain a plurality of intermediate effects;
for each intermediate effect, acquiring a second initial score corresponding to each of the first music emotion, the first music style and the first music theme according to an identification of the each intermediate effect; and
screening the plurality of intermediate effects according to the second initial score to obtain the plurality of candidate effects.

### Claim 20 (depends on 19)

The non-transitory computer-readable storage medium according to claim 19, wherein the screening the plurality of rendering effects according to the first initial score to obtain a plurality of intermediate effect, comprises:
for each rendering effect, determining a sum of the first initial score corresponding to each of the first image emotion, the first image style and the first image scene, as a first target score corresponding to the each rendering effect; and
determining rendering effects of which the first target score is greater than or equal to a first threshold in the plurality of rendering effects, as the plurality of intermediate effects.

### Claim 21 (depends on 19)

The non-transitory computer-readable storage medium according to claim 19, wherein the screening the plurality of intermediate effects according to the second initial score to obtain the plurality of candidate effects, comprises:
for each intermediate effect, determining a sum of the second initial score corresponding to each of the first music emotion, the first music style and the first music theme, as a second target score corresponding to the each intermediate effect; and
determining intermediate effects of which the second target score is greater than or equal to a second threshold in the plurality of intermediate effects, as the plurality of candidate effects.
