Continual learning for multi modal systems using crowd sourcing

A non-transitory computer-readable medium comprising instructions, when executed by a computing system, the instructions cause the computing system to:

• separate media data into one or more clusters, each cluster of the one or more clusters based on a feature from a first model;
• sample the media data of each cluster and, based on an analysis of the sampled media data, determine an accuracy of the media data of each cluster, the accuracy associated with the feature;
• based on a subset dataset of the media data being outside a threshold accuracy, automatically forward the subset dataset to a crowd source service;
• receive verification of the subset dataset from the crowd source service;
• and
• add the verified subset dataset to the first model.

A method of training a model comprising:

• separating media data into one or more clusters, each cluster of the one or more clusters based on a feature from a first model;
• sampling the media data of each cluster and, based on an analysis of the sampled media data, determining an accuracy of the mediadata of each cluster, the accuracy associated with the feature;
• based on a subset dataset of the media data being outside a threshold accuracy, automatically forwarding the subset dataset to a crowd source service;
• receiving verification of the subset dataset from the crowd source service;
• and
• adding the verified subset dataset to the first model.