System and method for multi-modal annotation

A non-transitory computer-readable medium having program code recorded thereon for multi-modal annotation, the program code being executed by a processor and comprising:

• program code to generate, using a training data generation model, a pre-annotated training data for human annotations;
• program code to iteratively verify the human annotations of the pre-annotated training data using a different annotator from a human annotator or the human annotations;
• program code to adjust the human annotations of the pre-annotated training data based on interactively verifying to finalize an annotated training data;
• and
• program code to train a machine learning model using the annotated training data.

A system for multi-modal annotation, the system comprising:

• a training data generation model to generate, using a training data generation model, a pre-annotated training data for human annotations;
• an annotated data verification model to iteratively verify the human annotations of the pre-annotated training data using a different annotator from a human annotator or the human annotations;
• an annotated data modification model to adjust the human annotations of the pre-annotated training data based on interactively verifying to finalize an annotated training data;
• and
• a model training module to train a machine learning model using the annotated training data.