Asynchronous content analysis for synchronizing audio and video streams

A system, comprising:

• a processor;
• and
• a memory, wherein the memory includes a computer program product configured to perform operations for synchronizing an unaligned audio stream and a corresponding unaligned video stream of real time streaming media, the operations comprising:
• performing, using a video classifier, video reference point classification of a video stream based on an audio-video dataset;
• wherein the audio-video dataset comprises audio-video data of objects causing a Doppler-effect sound in a corresponding audio stream;
• performing, using an audio classifier, audio reference point classification of the corresponding audio stream based on the audio-video dataset;
• identifying, based on the video reference point classification, a set of video segments comprising object related reference points in the video stream;
• wherein object related reference points in the video stream include the Doppler-effect sound in the audio stream;
• identifying, based on the audio reference point classification, a set of audio segments comprising object related reference points of the Doppler-effect sound in the audio stream;
• correlating object related reference points in the video segments of the video stream and in the audio segments of the audio stream to identify a set of audio-video synchronization candidates;
• and
• comparing context of the set of the audio-video synchronization candidates to identify an audio-video synchronization candidate to synchronize the audio stream and the video stream.

A computer program product for synchronizing an unaligned audio stream and a corresponding unaligned video stream of real time streaming media, the computer program product comprising:

• a computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code executable by one or more computer processors to perform an operation comprising:
• performing, using a video classifier, video reference point classification of a video stream based on an audio-video dataset;
• wherein the audio-video dataset comprises audio-video data of objects causing a Doppler-effect sound in a corresponding audio stream;
• performing, using an audio classifier, audio reference point classification of the corresponding audio stream based on the audio-video dataset;
• identifying, based on the video reference point classification, a set of video segments comprising object related reference points in the video stream;
• wherein object related reference points in the video stream include the Doppler-effect sound in the audio stream;
• identifying, based on the audio reference point classification, a set of audio segments comprising object related reference points of the Doppler-effect sound in the audio stream;
• correlating object related reference points in the video segments of the video stream and in the audio segments of the audio stream to identify a set of audio-video synchronization candidates;
• and
• comparing context of the set of the audio-video synchronization candidates to identify an audio-video synchronization candidate to synchronize the audio stream and the video stream.