Method and apparatus for pre-training semantic representation model and electronic device

An electronic device, comprising:

• at least one processor;
• and
• a memory communicatively coupled to the at least one processor;
• wherein, the memory stores instructions executable by the at least one processor, when the instructions are executed by the at least one processor, the at least one processor configured to:
• for each video-text pair in pre-training data, determine a mask image sequence, a mask character sequence, and a mask image-character sequence of the video-text pair;
• determine a plurality of feature sequences and mask position prediction results respectively corresponding to the plurality of feature sequences by inputting the mask image sequence, the mask character sequence, and the mask image-character sequence into an initial semantic representation model;
• and
• build a loss function based on the plurality of feature sequences, the mask position prediction results, and true mask position results, and adjust coefficients of the semantic representation model to realize training.

A non-transitory computer-readable storage medium having computer instructions stored thereon, wherein the computer instructions are configured to cause a computer to implement a method for pre-training a semantic representation model, the method comprising:

• for each video-text pair in pre-training data, determining a mask image sequence, a mask character sequence, and a mask image-character sequence of the video-text pair;
• determining a plurality of feature sequences and mask position prediction results respectively corresponding to the plurality of feature sequences by inputting the mask image sequence, the mask character sequence, and the mask image-character sequence into an initial semantic representation model;
• and
• building a loss function based on the plurality of feature sequences, the mask position prediction results, and true mask position results, and adjusting coefficients of the semantic representation model to realize training.