Multimodal speech recognition for real-time video audio-based display indicia application

A system to automatically generate audio-based display indicia of media content comprising:

• a memory having computer readable instructions;
• and
• a processor configured to execute the computer readable instructions, the computer readable instructions comprising:
• defining, by the processor, a plurality of media content categories for media content by at least applying a non-supervised clustering algorithm based at least in part on a set of features extracted from the media content, the set of features comprising metadata features that are extracted from one or more video descriptions of the media content, image features extracted from one or more images of the media content, drawing and reality features extracted from one or more images of the media content, and recognized characteristics of characters of the media content;
• receiving, by the processor, a plurality of categorized speech recognition algorithms, each speech recognition algorithm being associated with a respective one or more of the plurality of media content categories;
• determining, by the processor, a media content category of a current media content from the plurality of media content categories based at least in part on a set of current features extracted from the current media content, the set of current features comprising metadata features extracted from one or more video descriptions of the current media content, image features extracted from the current media content, drawing and reality features extracted from the current media content, and recognized characteristics of characters of the current media content;
• selecting, by the processor, one speech recognition algorithm from the plurality of categorized speech recognition algorithms based on the determination of the media content category of the current media content;
• and
• applying, by the processor the selected speech recognition algorithm to the current media content.

A computer program product to automatically generate audio-based display indicia of media content, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, the program instructions executable by a processor to cause the processor to:

• define, by the processor, a plurality of media content categories for media content by at least applying a non-supervised clustering algorithm based at least in part on a set of features extracted from the media content, the set of features comprising metadata features that are extracted from one or more video descriptions of the media content, image features extracted from one or more images of the media content, drawing and reality features extracted from one or more images of the media content, and recognized characteristics of characters of the media content;
• receive, by the processor, a plurality of categorized speech recognition algorithms, each speech recognition algorithm being associated with a respective one or more of the plurality of media content categories;
• determine, by the processor, a media content category of a current media content from the plurality of media content categories based at least in part on a set of current features extracted from the current media content, the set of current features comprising metadata features extracted from one or more video descriptions of the current media content, image features extracted from the current media content, drawing and reality features extracted from the current media content, and recognized characteristics of characters of the current media content;
• select, by the processor, one speech recognition algorithm from the plurality of categorized speech recognition algorithms based on the determination of the media content category of the current media content;
• and
• apply, by the processor, the selected speech recognition algorithm to the current media content.