Generating digital video summaries utilizing aesthetics, relevancy, and generative neural networks

A non-transitory computer readable medium for generating digital summaries of digital videos, the non-transitory computer readable medium comprising instructions that, when executed by at least one processor, cause a computer device to:

• apply a first convolutional neural network to a plurality of frames of a digital video to generate a set of deep features for the plurality of frames;
• utilize a relevancy neural network comprising a second convolutional neural network and a long short-term memory network to select one or more frames of the digital video based on the set of deep features;
• apply a generative reconstructor neural network comprising a long short-term memory encoder and a long short-term memory decoder to a subset of deep features corresponding to the one or more selected frames of the digital video to generate a digital video reconstruction;
• compare the digital video reconstruction to the digital video utilizing a loss function;
• and
• generate a digital video summary from the one or more selected frames of the digital video based on the comparison of the digital video reconstruction to the digital video.

A system comprising:

• at least one processor;
• and
• a memory comprising a digital video summary model, wherein the digital video summary model comprises:
• a relevancy neural network comprising a convolutional neural network and a long short-term memory network;
• and
• a generative reconstructor neural network comprising a long short-term memory encoder and a long short-term memory decoder;
• and
• the digital video summary model is operable to generate a plurality of digital video summary formats comprising at least two of:
• a thumbnail format, a storyboard format, an animated thumbnail format, and a video trailer format;
• and
• instructions that, when executed by the at least one processor, cause the system to:
• identify an indication of a digital video summary format of the plurality of video summary formats to generate for a digital video;
• utilize the relevancy neural network to select one or more frames of the digital video;
• apply the generative reconstructor neural network to the selected one or more frames of the digital video to generate a digital video reconstruction;
• compare the digital video reconstruction to the digital video utilizing a loss function;
• and
• generate, from the selected one or more frames, a digital video summary in the identified digital video summary format based on the comparison of the digital video reconstruction to the digital video.