# US 12,608,851 B2 · Method, apparatus, device and computer program product for generating video

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,608,851 B2
- Document ID: US12608851B2
- Status: Granted
- Kind: B2
- Application number: 18/774,689
- Filed: 2024-07-16
- Published: 2025-07-31
- Granted: 2026-04-21
- Earliest claimed: 2024-01-30
- Assignees: Beijing Youzhuju Network Technology Co., Ltd.
- Applicants: Beijing Youzhuju Network Technology Co., Ltd.
- Inventors: Jiawei Wang; Yuchen Zhang; Jiaxin Zou; Yan Zeng; Guoqiang Wei; Liping Yuan; Hang Li
- Web page: https://patentgrep.com/patent/US12608851B2

## CPC

- G06T 11/00: Two-dimensional [2D] image generation
- G11B 27/031: Electronic editing of digitised analogue information signals, e.g. audio or video signals
- G06F 40/289: Phrasal analysis, e.g. finite state techniques or chunking
- G06T 13/80: Two-dimensional [2D] animation, e.g. using sprites
- H04N 21/854: Content authoring
- G06T 2210/12: Bounding box
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

The present disclosure relates to a method and apparatus for generating a video, a device, and a computer program product. The method includes obtaining content information related to content of the video to be generated, where the content information includes at least one of a text or an image. The method further includes obtaining position information indicating a position of an object in the video in a starting frame. The method also includes obtaining control information that constrains a position of the object in an ending frame. In addition, the method further includes generating the video based on the content information, the position information, and the control information.

## Claims

### Claim 1

A method for generating a video, comprising:
obtaining content information related to content of the video to be generated, wherein the content information comprises a starting frame of the video;
obtaining, via a user interface, position information input by a user for identifying an object whose motion is to be controlled in the starting frame, wherein the position information is obtained through a first user interaction on the starting frame;
obtaining, via the user interface, control information constraining a position of the object in an ending frame, wherein the control information comprises at least one of a drawn bounding box or a drawn motion trajectory, and the control information is obtained through a second user interaction on the starting frame or a component representing the ending frame; and
generating the video based on the content information, the position information, and the control information,
wherein generating the video based on the content information, the position information, and the control information comprises:
in response to the control information being a trajectory line drawn in the starting frame, generating the video by controlling the object to move from a starting position along the trajectory line to a position indicated by an end point of the trajectory line.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the object is a first object, the position information is a first bounding box with a first color, the control information is first control information with the first color, and generating the video comprises:
obtaining a second bounding box indicating a position of a second object in the video in the starting frame, wherein the second bounding box has a second color different from the first color;
obtaining second control information constraining a position of the second object in the ending frame, wherein the second control information has the second color; and
generating the video based on the content information, the first bounding box, the first control information, the second bounding box, and the second control information.

### Claim 3 (depends on 1)

The method according to claim 1, wherein the position information is a starting bounding box, the control information is an ending bounding box for the object in the ending frame, and both the starting bounding box and the ending bounding box are rectangular boxes.

### Claim 4 (depends on 3)

The method according to claim 3, wherein generating the video based on the content information, the position information, and the control information comprises:
generating, in response to the user selecting a first type as a type of the ending bounding box, the video by making the object move from a position indicated by the starting bounding box to a specific position indicated by the ending bounding box, wherein a size of the object in the ending frame corresponds to the ending bounding box.

### Claim 5 (depends on 3)

The method according to claim 3, wherein generating the video based on the content information, the position information, and the control information comprises:
generating, in response to the user selecting a second type as a type of the ending bounding box, the video by making the object move from a position indicated by the starting bounding box to a position range indicated by the ending bounding box, wherein a size of the object in the ending frame does not exceed the ending bounding box, and content of the video is associated with the content information.

### Claim 6 (depends on 3)

The method according to claim 3, wherein generating the video based on the content information, the position information, and the control information comprises:
generating, in response to the ending bounding box being close to a left boundary or a right boundary of the ending frame and a width of the ending bounding box being less than a threshold width, the video by making the object move from a position in the starting frame to a position beyond the left boundary or the right boundary of the ending frame, wherein a size of the object as the object moves beyond the left boundary or the right boundary is associated with a height of the ending bounding box; or,
generating, in response to the ending bounding box being close to an upper boundary or a lower boundary of the ending frame and the height of the ending bounding box being less than a threshold height, the video by making the object move from a position in the starting frame to a position beyond the upper boundary or the lower boundary of the ending frame, wherein a size of the object as the object moves beyond the upper boundary or the lower boundary is associated with the width of the ending bounding box, and
wherein content of the video is associated with the content information.

### Claim 7 (depends on 3)

The method according to claim 3, wherein generating the video based on the content information, the position information, and the control information comprises:
generating, in response to the starting bounding box being close to a left boundary or a right boundary of a starting frame and a width of the starting bounding box being less than a threshold width, the video by making the object move from a position beyond the left boundary or the right boundary to a position constrained by the control information, wherein a size of the object as the object enters the left boundary or the right boundary is associated with a height of the starting bounding box; or
generating, in response to the starting bounding box being close to an upper boundary or a lower boundary of the starting frame and the height of the starting bounding box being less than a threshold height, the video by making the object move from a position beyond the upper boundary or the lower boundary of the starting frame to a position constrained by the control information, wherein a size of the object as the object enters the upper boundary or the lower boundary is associated with the width of the ending bounding box, and
wherein content of the video is associated with the content information.

### Claim 8 (depends on 1)

The method according to claim 1, wherein the position information is a bounding box.

### Claim 9 (depends on 8)

The method according to claim 8, wherein generating the video based on the content information, the position information, and the control information comprises:
generating the video by making the object start to move from a position indicated by the bounding box along the motion trajectory, wherein content of the video is associated with the content information.

### Claim 10 (depends on 1)

The method according to claim 1, wherein the position information is a bounding box, and generating the video based on the content information, the position information, and the control information comprises:
determining image content identified by the bounding box in the starting frame as the object; and
generating the video based on an image of the starting frame, the object, and the control information, wherein content of the video is associated with the image.

### Claim 11 (depends on 10)

The method according to claim 10, wherein the content information comprises a text inputted by the user for describing the content of the video, and the method further comprises:
recognizing a noun phrase in the text;
determining the noun phrase as the object; and
generating the video based on the text, the object, and the control information, wherein the content of the video is associated with the text.

### Claim 12 (depends on 1)

The method according to claim 1, wherein the control information is first control information, and the method further comprises:
obtaining third control information constraining a position of the object in an intermediate frame; and
generating the video based on the content information, the position information, the first control information, and the third control information.

### Claim 13 (depends on 1)

The method according to claim 1, wherein the object gradually moves from a first position to a second position in the video.

### Claim 14 (depends on 1)

The method according to claim 1, wherein in the video, by changing a perspective of a camera, the object moves relative to the camera.

### Claim 15

An electronic device, comprising:
a processor; and
a memory coupled with the processor, wherein the memory has instructions stored therein, and the instructions, when executed by the processor, cause the electronic device to:
obtain content information related to content of a video to be generated, wherein the content information comprises a starting frame of the video;
obtain, via a user interface, position information input by a user for identifying an object whose motion is to be controlled in the starting frame, wherein the position information is obtained through a first user interaction on the starting frame;
obtain, via the user interface, control information constraining a position of the object in an ending frame, wherein the control information comprises at least one of a drawn bounding box or a drawn motion trajectory, and the control information is obtained through a second user interaction on the starting frame or a component representing the ending frame; and
generate the video based on the content information, the position information, and the control information,
wherein the instructions causing the electronic device to generate the video based on the content information, the position information, and the control information comprise instructions causing the electronic device to:
in response to the control information being a trajectory line drawn in the starting frame, generate the video by controlling the object to move from a starting position along the trajectory line to a position indicated by an end point of the trajectory line.

### Claim 16 (depends on 15)

The electronic device according to claim 15, wherein the object is a first object, the position information is a first bounding box with a first color, the control information is first control information with the first color, and the instructions causing the electronic device to generate the video based on the content information, the position information, and the control information comprise instructions causing the electronic device to:
obtain a second bounding box indicating a position of a second object in the video in the starting frame, wherein the second bounding box has a second color different from the first color;
obtain second control information constraining a position of the second object in the ending frame, wherein the second control information has the second color; and
generate the video based on the content information, the first bounding box, the first control information, the second bounding box, and the second control information.

### Claim 17 (depends on 15)

The electronic device according to claim 15, wherein the position information is a starting bounding box, the control information is an ending bounding box for the object in the ending frame, and both the starting bounding box and the ending bounding box are rectangular boxes.

### Claim 18 (depends on 17)

The electronic device according to claim 17, wherein the instructions causing the electronic device to generate the video based on the content information, the position information, and the control information comprise instructions causing the electronic device to:
generate, in response to the user selecting a first type as a type of the ending bounding box, the video by making the object move from a position indicated by the starting bounding box to a specific position indicated by the ending bounding box, wherein a size of the object in the ending frame corresponds to the ending bounding box.

### Claim 19 (depends on 17)

The electronic device according to claim 17, wherein the instructions causing the electronic device to generate the video based on the content information, the position information, and the control information comprise instructions causing the electronic device to:
generate, in response to the user selecting a second type as a type of the ending bounding box, the video by making the object move from a position indicated by the starting bounding box to a position range indicated by the ending bounding box, wherein a size of the object in the ending frame does not exceed the ending bounding box, and content of the video is associated with the content information.

### Claim 20

A computer program product, wherein the computer program product is tangibly stored on a non-transitory computer-readable medium and comprises machine-executable instructions, and the machine-executable instructions, when executed, cause a machine to:
obtain content information related to content of a video to be generated, wherein the content information comprises a starting frame of the video;
obtain, via a user interface, position information input by a user for identifying an object whose motion is to be controlled in the starting frame, wherein the position information is obtained through a first user interaction on the starting frame;
obtain, via the user interface, control information constraining a position of the object in an ending frame, wherein the control information comprises at least one of a drawn bounding box or a drawn motion trajectory, and the control information is obtained through a second user interaction on the starting frame or a component representing the ending frame; and
generate the video based on the content information, the position information, and the control information,
wherein the instructions causing the machine to generate the video based on the content information, the position information, and the control information comprise instructions causing the machine to:
in response to the control information being a trajectory line drawn in the starting frame, generate the video by controlling the object to move from a starting position along the trajectory line to a position indicated by an end point of the trajectory line.
