# US 2025/0061922 A1 · Generating videos

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0061922 A1
- Document ID: US20250061922A1
- Status: Published application
- Kind: A1
- Application number: 18/937,679
- Filed: 2024-11-05
- Published: 2025-02-20
- Earliest claimed: 2021-07-16
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Nathan James Frey; Zheng Sun
- Web page: https://patentgrep.com/patent/US20250061922A1

## CPC

- G11B 27/031: Electronic editing of digitised analogue information signals, e.g. audio or video signals
- G11B 27/28: by using information signals recorded by the same method as the main recording {(G11B27/22 takes precedence)}
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)}
- G06V 20/49: {Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes}
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06T 2207/10016: Video; Image sequence
- G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating videos. In one aspect, a method comprises: receiving: (i) an input video comprising a sequence of video frames, and (ii) data indicating a target object type; processing the input video to generate tracking data that identifies and tracks visual locations of one or more instances of target objects of the target object type in the input video; generating a plurality of sub-videos based on the input video and the tracking data, including: for each sub-video, generating a respective sequence of sub-video frames that are each extracted from a respective video frame of the input video to include a respective instance of a given target object from among the identified target objects of the target object type; and generating an output video that comprises the plurality of sub-videos.

## Claims

### Claim 1

A method performed by one or more computers, the method comprising:
receiving an input video comprising a sequence of video frames;
processing the input video to generate tracking data that identifies and tracks visual locations of one or more instances of at least one target object in the input video;
generating a set of multiple sub-videos based on the input video and the tracking data;
generating an output video that comprises a plurality of sub-videos from the set of multiple sub-videos; and
causing presentation of the output video at a user interface of a computing device.

### Claim 2 (depends on 1)

The method of claim 1, wherein generating the set of multiple sub-videos based on the input video and the tracking data comprises, for each sub-video:
generating a respective sequence of sub-video frames that are each extracted from a respective video frame of the input video.

### Claim 3 (depends on 2)

The method of claim 2, wherein for each sub-video, each of the sub-video frames includes a depiction of the at least one target object and excludes at least some content of the respective video frame that is outside of a bounding box that encloses the target object in each of the sub-video frames.

### Claim 4 (depends on 1)

The method of claim 1, wherein each of the sub-videos has a duration that is shorter than the input video.

### Claim 5 (depends on 1)

The method of claim 1, wherein processing the input video to generate tracking data that identifies and tracks visual locations of one or more instances of at least one target object in the input video comprises:
processing each video frame in the sequence of video frames of the input video using an object detection neural network to generate, for each of a plurality of the video frames, a respective object detection output that defines one or more bounding boxes that each enclose a respective object in the video frame; and
generating the tracking data using the object detection outputs for the video frames in the sequence of video frames of the input video.

### Claim 6 (depends on 5)

The method of claim 5, wherein generating the tracking data using the object detection outputs for the video frames in the sequence of video frames in the input video comprises:
determining a similarity measure between a first bounding box in a first video frame and a second bounding box in a second video frame satisfies a threshold; and
in response, determining that the first bounding box and the second bounding box are included in a same target object trajectory that tracks a position of a target object from the first video frame to the second video frame.

### Claim 7 (depends on 6)

The method of claim 6, wherein determining the similarity measure between the first bounding box in the first video frame and the second bounding box in the second video frame comprises:
determining an overlap between the first bounding and the second bounding box; and
determining the similarity measure based at least in part on the overlap between the first bounding box and the second bounding box.

### Claim 8 (depends on 6)

The method of claim 6, wherein determining the similarity measure between the first bounding box in the first video frame and the second bounding box in the second video frame comprises:
determining a visual similarity between respective video frame content enclosed by the first bounding box and the second bounding box; and
determining the similarity measure based at least in part on the visual similarity between the respective video frame content enclosed by the first bounding box and the second bounding box.

### Claim 9 (depends on 1)

The method of claim 1, wherein generating the output video that comprises the plurality of sub-videos from the set of multiple sub-videos comprises:
selecting a proper subset of the set of multiple sub-videos for inclusion in the output video.

### Claim 10 (depends on 9)

The method of claim 9, wherein selecting the proper subset of the set of multiple sub-videos for inclusion in the output video comprises:
determining a respective score for each sub-video in the set of multiple sub-videos that characterizes a quality of the sub-video; and
selecting the proper subset of the set of multiple sub-videos based on the scores.

### Claim 11 (depends on 10)

The method of claim 10, wherein for each sub-video in the set of multiple sub-videos, determining the score for the sub-video comprises:
determining the score for the sub-video based at least in part on an amount of motion in the sub-video.

### Claim 12 (depends on 10)

The method of claim 10, wherein for each sub-video in the set of multiple sub-videos, determining the score for the sub-video comprises:
determining the score for the sub-video based at least in part on a sharpness of the sub-video.

### Claim 13 (depends on 1)

The method of claim 1, wherein causing presentation of the output video at the user interface of the computing device comprises:
receiving a request for a digital component;
determining that a digital component that includes the output video is responsive to the request; and
providing the digital component that includes the output video in response to the request to be presented alongside search results or on a third-party webpage.

### Claim 14

A system comprising:
one or more computers; and
one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising:
receiving an input video comprising a sequence of video frames;
processing the input video to generate tracking data that identifies and tracks visual locations of one or more instances of at least one target object in the input video;
generating a set of multiple sub-videos based on the input video and the tracking data;
generating an output video that comprises a plurality of sub-videos from the set of multiple sub-videos; and
causing presentation of the output video at a user interface of a computing device.

### Claim 15

One or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:
receiving an input video comprising a sequence of video frames;
processing the input video to generate tracking data that identifies and tracks visual locations of one or more instances of at least one target object in the input video;
generating a set of multiple sub-videos based on the input video and the tracking data;
generating an output video that comprises a plurality of sub-videos from the set of multiple sub-videos; and
causing presentation of the output video at a user interface of a computing device.

### Claim 16 (depends on 15)

The one or more non-transitory computer storage media of claim 15, wherein generating the set of multiple sub-videos based on the input video and the tracking data comprises, for each sub-video:
generating a respective sequence of sub-video frames that are each extracted from a respective video frame of the input video.

### Claim 17 (depends on 16)

The one or more non-transitory computer storage media of claim 16, wherein for each sub-video, each of the sub-video frames includes a depiction of the at least one target object and excludes at least some content of the respective video frame that is outside of a bounding box that encloses the target object in each of the sub-video frames.

### Claim 18 (depends on 15)

The one or more non-transitory computer storage media of claim 15, wherein each of the sub-videos has a duration that is shorter than the input video.

### Claim 19 (depends on 15)

The one or more non-transitory computer storage media of claim 15, wherein processing the input video to generate tracking data that identifies and tracks visual locations of one or more instances of at least one target object in the input video comprises:
processing each video frame in the sequence of video frames of the input video using an object detection neural network to generate, for each of a plurality of the video frames, a respective object detection output that defines one or more bounding boxes that each enclose a respective object in the video frame; and
generating the tracking data using the object detection outputs for the video frames in the sequence of video frames of the input video.

### Claim 20 (depends on 19)

The one or more non-transitory computer storage media of claim 19, wherein generating the tracking data using the object detection outputs for the video frames in the sequence of video frames in the input video comprises:
determining a similarity measure between a first bounding box in a first video frame and a second bounding box in a second video frame satisfies a threshold; and
in response, determining that the first bounding box and the second bounding box are included in a same target object trajectory that tracks a position of a target object from the first video frame to the second video frame.
