Self-training on unpaired data for vision-language models

A non-transitory computer readable medium storing code for training a machine learning model, the code comprising instructions executable by at least one processor to perform operations comprising:

• obtaining training data including an input image;
• training, using the training data, a first captioning model to generate a synthetic caption based on the input image;
• generating an augmented caption based on the synthetic caption;
• and
• using the synthetic caption and the augmented caption to train a second captioning model.

An apparatus comprising:

• at least one processor;
• at least one memory component coupled with the at least one processor;
• and
• a captioning model comprising parameters stored in the at least one memory component and trained to generate a text caption describing an input image, wherein the captioning model is trained by generating a synthetic caption, generating an augmented caption based on the synthetic caption, and training the captioning model using the synthetic caption and the augmented caption.