Video classification using user behavior from a network digital video recorder

A system comprising:

• a broadcast delivery system configured to broadcast a video asset based on a broadcast schedule, wherein the video asset includes video program segments and ad segments;
• a network digital video recorder (nDVR) system coupled to the broadcast delivery system and configured to record the video asset to allow users to request the video using the nDVR system on demand;
• a classifier coupled to receive user behavior information from user devices viewing the video asset using the nDVR system, the classifier configured for:
• performing a content analysis of a baseline set of audio, video, and/or text (AVT) features of the video asset to obtain a boundary that defines an ending of a first video segment and a beginning of a second video segment;
• performing a user behavior analysis of the user behavior information received from the user devices viewing the video asset using the nDVR system during the first video segment and the second video segment;
• performing a content similarity analysis between first AVT features extracted over a first time period in content selected from the first video segment, the first time period having a duration in seconds, and second AVT features extracted over a second time period in content selected from the second video segment, the second time period having the same duration, the analysis comprising determining a content similarity over the duration of the first time period and the second time period, based at least on a similarity metric of first statistics for the first AVT features over the duration to second statistics for the second AVT features over the duration;
• classifying the first video segment and the second video segment, by using a fusion of results from both the content analysis and the user behavior analysis, to obtain a classification for each video segment that identifies the video segment as either an ad segment or a video program segment based on the fusion of results from both the content analysis and the user behavior analysis, wherein a fusion function performs the fusion of results using both user behavior and content analysis simultaneously to classify a video segment as an ad segment;
• checking a classification of a video segment as an ad segment by using the content similarity analysis to obtain a determination of whether the classification is a false positive result;
• based on the determination, changing the classification of the video segment if the classification as an ad segment is a false positive result;
• and
• determining an action to be performed on the video asset based on the classifying of the first video segment and the second video segment.