Content recognition method and apparatus, computer device, and storage medium

An apparatus for content recognition, comprising:

• processing circuitry configured to:
• acquire, from a content for recognition, a text piece and a media piece associated with the text piece, the media piece including image data or audio data of the content;
• perform a first feature extraction on the text piece to obtain text features;
• perform a second feature extraction on the media piece associated with the text piece to obtain media features;
• determine feature association measures between the media features and the text features, a feature association measure for a first feature in the media features and a second feature in the text features indicating an association degree between the first feature and the second feature;
• obtain feature attention intensities corresponding to the text features based on the feature association measures, each value of the feature association measures being positively correlated with a respective value of the feature attention intensities, and the feature attention intensities including image attention intensities based on the image data or audio attention intensities based on the audio data;
• adjust the text features based on the feature association measures to obtain adjusted text features that include first adjusted text features obtained by adjusting the text features based on the image attention intensities or second adjusted text features obtained by adjusting the text features based on the audio attention intensities;
• and
• perform a recognition based on the adjusted text features to obtain a content recognition result of the content.

A non-transitory computer-readable storage medium storing instructions which, when executed by at least one processor, cause the at least one processor to perform a method for content recognition comprising:

• acquiring, from a content for recognition, a text piece and a media piece associated with the text piece, the media piece including image data or audio data of the content;
• performing a first feature extraction on the text piece to obtain text features;
• performing a second feature extraction on the media piece associated with the text piece to obtain media features;
• determining feature association measures between the media features and the text features, a feature association measure for a first feature in the media features and a second feature in the text features indicating an association degree between the first feature and the second feature;
• obtaining feature attention intensities corresponding to the text features based on the feature association measures, each value of the feature association measures being positively correlated with a respective value of the feature attention intensities, and the feature attention intensities including image attention intensities based on the image data or audio attention intensities based on the audio data;
• adjusting the text features based on the feature association measures to obtain adjusted text features that include first adjusted text features obtained by adjusting the text features based on the image attention intensities or second adjusted text features obtained by adjusting the text features based on the audio attention intensities;
• and
• performing a recognition based on the adjusted text features to obtain a content recognition result of the content.