Intelligent cataloging method for all-media news based on multi-modal information fusion understanding

An intelligent cataloging method for all-media news based on multi-modal information fusion understanding, comprising the following steps:

• 1) obtaining original news video, segmenting shot fragments, and locating scene key frames;
• 2) inferring scene classification labels from the scene key frames obtained in step 1) and merging adjacent shot fragments with similar scene labels to generate multiple slice fragments;
• 3) performing visual feature extraction on the slice fragments obtained in step 2) and generating news description text;
• 4) performing voice recognition on the slice fragments obtained in step 2) to obtain voice text;
• 5) extracting image frame recognition of the slice fragments obtained in step 2) to obtain subtitle bar text;
• 6) recognizing facial features in the slice fragments obtained in step 2) and matching the facial features in a news character database to obtain character information text;
• and
• 7) inputting the news description text obtained in step 3), the voice text obtained in step 4), the subtitle bar text obtained in step 5), and the character information text obtained in step 6) into a generative model of multi-modal fusion for processing to generate news keywords and comprehensive cataloging descriptions, and performing outputting after sorting and assembly to complete intelligent cataloging for the news.