Video caption generating method and apparatus, device, and storage medium

A video caption generating apparatus, disposed in a computer device, the apparatus comprising:

• a memory storing computer program instructions;
• and a processor coupled to the memory and configured to execute the computer program instructions and perform:
• encoding a target video by using an encoder of a video caption generating model, to obtain a target visual feature of the target video;
• decoding the target visual feature by using a basic decoder of the video caption generating model, to obtain a first selection probability corresponding to a candidate word;
• decoding the target visual feature by using an auxiliary decoder of the video caption generating model, to obtain a second selection probability corresponding to the candidate word, a memory structure of the auxiliary decoder including reference visual context information corresponding to the candidate word, and the reference visual context information being generated according to a related video corresponding to the candidate word;
• determining a decoded word in the candidate word according to the first selection probability and the second selection probability;
• and
• generating a video caption corresponding to the target video according to each decoded word.

A non-transitory computer-readable storage media, storing at least one computer-readable instruction, at least one program, a code set, or a computer-readable instruction set, the at least one computer-readable instruction, the at least one program, the code set, or the computer-readable instruction set being loaded and executed by one or more processors to perform:

• encoding a target video by using an encoder of a video caption generating model, to obtain a target visual feature of the target video;
• decoding the target visual feature by using a basic decoder of the video caption generating model, to obtain a first selection probability corresponding to a candidate word;
• decoding the target visual feature by using an auxiliary decoder of the video caption generating model, to obtain a second selection probability corresponding to the candidate word, a memory structure of the auxiliary decoder including reference visual context information corresponding to the candidate word, and the reference visual context information being generated according to a related video corresponding to the candidate word;
• determining a decoded word in the candidate word according to the first selection probability and the second selection probability;
• and
• generating a video caption corresponding to the target video according to decoded word.