Method and apparatus for training medical image report generation model, and image report generation method and apparatus

A training apparatus for a medical image report generation model, the medical image report generation model comprising a visual feature extraction network, an encoding network, and a decoding network, the apparatus comprising:

• a processor and a memory, the memory storing at least one instruction, at least one program, a code set, or an instruction set, the at least one instruction, the at least one program, the code set, or the instruction set being loaded and executed by the processor to implement:
• acquiring a sample medical image;
• extracting visual feature information of the sample medical image through the visual feature extraction network, to obtain a visual feature sequence of the sample medical image;
• concatenating a self-learning label based on the visual feature sequence, to obtain input information about the encoding network;
• encoding the input information through the encoding network, to obtain a visual encoding feature vector corresponding to the visual feature sequence and an output task result corresponding to the self-learning label;
• decoding the visual encoding feature vector through the decoding network, to obtain an output image report corresponding to the sample medical image;
• calculating a total loss function value of the medical image report generation model based on a first loss function value and a third loss function value, the first loss function value being calculated based on the output image report and a target image report corresponding to the sample medical image, and the third loss function value being calculated based on the output task result and a target task result corresponding to the sample medical image;
• and
• adjusting a parameter of the medical image report generation model according to the total loss function value.

A non-transitory computer readable storage medium, storing a computer program for training a medical image report generation model, the medical image report generation model comprising a visual feature extraction network, an encoding network, and a decoding network, the computer program, when being executed by a processor of a computer device, causing the processor to implement:

• acquiring a sample medical image;
• extracting visual feature information of the sample medical image through the visual feature extraction network, to obtain a visual feature sequence of the sample medical image;
• concatenating a self-learning label based on the visual feature sequence, to obtain input information about the encoding network;
• encoding the input information through the encoding network, to obtain a visual encoding feature vector corresponding to the visual feature sequence and an output task result corresponding to the self-learning label;
• decoding the visual encoding feature vector through the decoding network, to obtain an output image report corresponding to the sample medical image;
• calculating a total loss function value of the medical image report generation model based on a first loss function value and a third loss function value, the first loss function value being calculated based on the output image report and a target image report corresponding to the sample medical image, and the third loss function value being calculated based on the output task result and a target task result corresponding to the sample medical image;
• and
• adjusting a parameter of the medical image report generation model according to the total loss function value.