US 11,769,327 B2 · Automatically and precisely generating highlight videos with artificial intelligence

Abstract

Presented herein are systems, methods, and datasets for automatically and precisely generating highlight or summary videos of content. For example, in one or more embodiments, videos of sporting events may be digested or condensed into highlights, which will dramatically benefit sports media, broadcasters, video creators or commentators, or other short video creators, in terms of cost reduction, fast, and mass production, and saving tedious engineering hours. Embodiment of the framework may also be used or adapted for use to better promote sports teams, players, and/or games, and produce stories to glorify the spirit of sports or its players. While presented in the context of sports, it shall be noted that the methodologies may be used for videos comprising other content and events.

Claims20

Dependent claims

A computer-implemented method comprising:

for each video from a set of one or more videos:
performing time anchoring to correlate video runtime with timing of a happening captured in the video;
generating a clip from the video that includes an event of interest by using metadata related to the happening and correlated time obtained by time anchoring to identify an approximate time of the event of interest that occurred at the happening;
performing feature extraction on the clip;
and
obtaining a final temporal value of the event of interest in the clip using the extracted features and neural network model;
for each clip from a set of clips generated from the set of one or more videos, comparing the final temporal value with a corresponding ground-truth value to obtain a loss value;
and
using the loss values to update the neural network model.

6 more elements

A system comprising:

one or more processors;
and
a non-transitory computer-readable medium or media comprising one or more sets of instructions which, when executed by at least one of the one or more processors, causes steps to be performed comprising:
for each video from a set of one or more videos:
performing time anchoring to correlate video runtime with timing of a happening captured in the video;
generating a clip from the video that includes an event of interest by using metadata related to the happening and correlate time obtained by time anchoring to identify an approximate time of the event of interest that occurred at the happening;
performing feature extraction on the clip;
and
obtaining a final temporal value of the event of interest in the clip using the extracted features and a neural network model;
and
using the final temporal value of the event of interest to make shorter clip that includes the event of interest.

8 more elements

A computer-implemented method comprising:

for each video from a set of one or more videos:
performing time anchoring to correlate video runtime with timing of a happening captured in the video;
generating a clip from the video that includes an event of interest by using metadata related to the happening and correlate time obtained by time anchoring to identify an approximate time of the event of interest that occurred at the happening;
performing feature extraction on the clip;
and
obtaining a final temporal value of the event of interest in the clip using the extracted features and neural network model;
and
using the final temporal value of the event of interest to make shorter clip that includes the event of interest.

5 more elements

Dependencies are inferred from the claim text.

Automatically and precisely generating highlight videos with artificial intelligence

Life of this patent

Automatically and precisely generating highlight videos with artificial intelligence

Life of this patent

Abstract

Claims20