Systems and methods for automatically generating sound event subtitles

A system comprising:

• at least one physical processor;
• and
• physical memory comprising computer-executable instructions that, when executed by the at least one physical processor, cause the at least one physical processor to perform acts comprising:
• identifying an input audio waveform from a soundtrack of a digital video;
• applying a sound event classification model to the input audio waveform to:
• detect one or more sound events within the input audio waveform, each of the one or more sound events comprising sounds other than speech;
• and
• generate sound event classification labels and sound event time boundaries for the one or more sound events;
• translating the sound event classification labels into digital video display metadata describing the one or more sound events;
• and
• maintaining the digital video display metadata in association with the digital video for playback within the digital video corresponding to the sound event time boundaries.

A computer-implemented method comprising:

• generating an input audio waveform from a soundtrack of a digital video;
• applying, in parallel:
• a sound event classification model to the input audio waveform to generate sound event classification labels;
• and
• a speech music activation detection model to the input audio waveform to generate waveform segments comprising speech and waveform segments comprising music;
• applying, in parallel:
• a music classification model to the waveform segments comprising music to generate music classification predictions for the waveform segments comprising music;
• and
• a lyric recognition model to the waveform segments comprising music to generate lyric transcriptions for the waveform segments comprising music;
• and
• generating a subtitle file associated with the digital video comprising one or more of the sound event classification labels, the music classification predictions, and the lyric transcriptions.