Model training method and apparatus, computer device, and storage medium

A computer device, comprising a memory and a processor, the memory storing a plurality of instructions, and the processor being configured to execute the plurality of instructions in the memory and cause the computer device to perform a video content retrieval method including:

• obtaining a query text from a user, wherein the query text is associated with a plurality of candidate videos;
• performing feature extraction processing on the query text through a text encoder in a video content retrieval model, to obtain a plurality of text content features at different feature granularities, wherein the plurality of text content features at the different feature granularities include a first text content feature at a coarse granularity and a second text content feature at a fine granularity;
• transcribing the plurality of candidate video based on optical character recognition (OCR) and automatic speech recognition (ASR) technology to obtain a plurality of speech features and text features from the plurality of candidate videos;
• performing multi-modal feature extraction processing on the plurality of candidate videos and their corresponding plurality of speech features and text features through a video encoder in the video content retrieval model, to obtain a plurality of multi-modal video content features at different feature granularities, wherein the plurality of multi-modal video content features at the different feature granularities include a first multi-modal video content feature at a coarse granularity and a second multi-modal video content feature at a fine granularity;
• calculating, based on the text content feature of each feature granularity, a similarity corresponding to the query text and a candidate video at the corresponding feature granularity, wherein the candidate video a first multi-modal video content feature at the coarse granularity and a second multi-modal video content feature at the fine granularity, further including:
• performing cross-modal alignment on the coarse granularity between the first text content feature and the first multi-modal video content feature to obtain a similarity between the query text and the candidate video corresponding to the coarse granularity;
• and
• performing cross-modal alignment on the fine granularity between the second text content feature and the second multi-modal video content feature to obtain a similarity between the query text and the candidate video content retrieval result corresponding to the fine granularity;
• and
• determining, based on the similarities between the query text and the plurality of candidate videos at different feature granularities including the coarse granularity and the fine granularity, one of the plurality of candidate videos having a maximum similarity as a video content retrieval result corresponding to the query text.

A non-transitory computer-readable storage medium, storing a plurality of instructions, and the instructions being configured to be loaded by a processor of a computer device, and causing the computer device to perform a video content retrieval method including:

• obtaining a query text from a user, wherein the query text is associated with a plurality of candidate videos;
• performing feature extraction processing on the query text through a text encoder in a video content retrieval model, to obtain a plurality of text content features at different feature granularities, wherein the plurality of text content features at the different feature granularities include a first text content feature at a coarse granularity and a second text content feature at a fine granularity;
• transcribing the plurality of candidate video based on optical character recognition (OCR) and automatic speech recognition (ASR) technology to obtain a plurality of speech features and text features from the plurality of candidate videos;
• performing multi-modal feature extraction processing on the plurality of candidate videos and their corresponding plurality of speech features and text features through a video encoder in the video content retrieval model, to obtain a plurality of multi-modal video content features at different feature granularities, wherein the plurality of multi-modal video content features at the different feature granularities include a first multi-modal video content feature at a coarse granularity and a second multi-modal video content feature at a fine granularity;
• calculating, based on the text content feature of each feature granularity, a similarity corresponding to the query text and a candidate video at the corresponding feature granularity, wherein the candidate video a first multi-modal video content feature at the coarse granularity and a second multi-modal video content feature at the fine granularity, further including:
• performing cross-modal alignment on the coarse granularity between the first text content feature and the first multi-modal video content feature to obtain a similarity between the query text and the candidate video corresponding to the coarse granularity;
• and
• performing cross-modal alignment on the fine granularity between the second text content feature and the second multi-modal video content feature to obtain a similarity between the query text and the candidate video content retrieval result corresponding to the fine granularity;
• and
• determining, based on the similarities between the query text and the plurality of candidate videos at different feature granularities including the coarse granularity and the fine granularity, one of the plurality of candidate videos having a maximum similarity as a video content retrieval result corresponding to the query text.