A method for fine-tuning a multimodal large language model machine to autonomously generate educational level-specific feedback in scene description tasks with respect to different educational stages, wherein the multimodal large language model machine is based on a vision encoder, a vision-language connector and a foundational large language model device, and the method comprising:
- constructing a multimodal training dataset comprising image-text-feedback samples, wherein each image-text-feedback sample includes an image sample, scene-description text sample matched with the image sample, and a corresponding educational level-specific feedback sample with respect to an educational stage;
- and
- fine-tuning the multimodal large language model machine with the constructed multimodal training dataset to generate educational level-specific feedback in scene description tasks by:
- training the vision encoder to extract visual features of an input image;
- training the vision-language connector to project the visual features extracted by the vision encoder into word embedding space of the foundational large language model device to obtain a sequence of visual tokens;
- and
- training the foundational large language model device to generate educational level-specific feedback based on the obtained sequence of visual tokens.