Method and apparatus for detecting similarity between multimedia information, electronic device, and storage medium

A multimedia information processing apparatus, comprising:

• a memory storing computer program instructions;
• and a processor coupled to the memory and configured to execute the computer program instructions and perform:
• determining a first audio feature vector for source multimedia information;
• determining a second audio feature vector for target multimedia information;
• determining a similarity between the source multimedia information and the target multimedia information by comparing the first audio vector and the second audio vector;
• wherein an audio feature vector for multimedia information is determined by:
• parsing the multimedia information to separate an audio from the multimedia information;
• converting the audio to obtain a mel spectrogram corresponding to the audio;
• determining input triplet samples based on the mel spectrogram;
• performing cross-processing on the input triplet samples through a convolutional layer and a max pooling layer of a multimedia information processing model, to obtain downsampling results of the input triplet samples;
• normalizing the downsampling results through a fully connected layer of the multimedia information processing model, to obtain normalized results;
• and
• performing deep factorization processing on the normalized results through the multimedia information processing model, to obtain the audio feature vector matching the input triplet samples.

A non-transitory computer-readable storage medium storing computer program instructions executable by at least one processor to perform:

• determining a first audio feature vector for source multimedia information;
• determining a second audio feature vector for target multimedia information;
• determining a similarity between the source multimedia information and the target multimedia information by comparing the first audio vector and the second audio vector;
• wherein an audio feature vector for multimedia information is determined by:
• parsing the multimedia information to separate an audio from the multimedia information;
• converting the audio to obtain a mel spectrogram corresponding to the audio;
• determining input triplet samples based on the mel spectrogram;
• performing cross-processing on the input triplet samples through a convolutional layer and a max pooling layer of a multimedia information processing model, to obtain downsampling results of the input triplet samples;
• normalizing the downsampling results through a fully connected layer of the multimedia information processing model, to obtain normalized results;
• and
• performing deep factorization processing on the normalized results through the multimedia information processing model, to obtain the audio feature vector matching the input triplet samples.