Automatic data curation

One or more non-transitory computer readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of:

• generating a set of outputs by processing at least one data sample using a trained machine learning model;
• concurrently determining on a plurality of computing instances a plurality of relevance scores between a plurality of sampling objectives and the set of outputs generated using the trained machine learning model, wherein each sampling objective in the plurality of sampling objectives is associated with a different computing instance in the plurality of computing instances and is associated with a different objective that is to be achieved during retraining of the trained machine learning model, and wherein each relevance score represents a relevance of a given sampling objective in the plurality of sampling objectives to the trained machine learning model;
• determining a given sampling objective for the data sample set that is to be achieved during retraining of the trained machine learning model, wherein the determining of the given sampling objective is based on the plurality of relevance scores;
• determining one or more data sampling criteria based on the given sampling objective;
• selecting, from a set of unlabeled data samples, at least one data sample for labeling and adding to the data sample set based on the one or more data sampling criteria;
• for each selected data sample, supplementing the data sample set with the selected data sample and at least one association with a label;
• and
• retraining the trained machine learning model using the supplemented data sample set.

A system, comprising:

• a memory that stores instructions, and
• a processor that is coupled to the memory and, when executing the instructions, is configured to:
• generate a set of outputs by processing at least one data sample using a trained machine learning model;
• concurrently determine on a plurality of computing instances a plurality of relevance scores between a plurality of sampling objectives and the set of outputs generated using the trained machine learning model, wherein each sampling objective in the plurality of sampling objectives is associated with a different computing instance in the plurality of computing instances and is associated with a different objective that is to be achieved during retraining of the trained machine learning model, and wherein each relevance score represents a relevance of a given sampling objective in the plurality of sampling objectives to the trained machine learning model;
• determine a given sampling objective for the data sample set that is to be achieved during retraining of the trained machine learning model, wherein the determining of the given sampling objective is based on the plurality of relevance scores;
• determine one or more data sampling criteria based on the given sampling objective;
• select, from a set of unlabeled data samples, at least one data sample for labeling and adding to the data sample set based on the one or more data sampling criteria;
• for each selected data sample, supplement the data sample set with the selected data sample and at least one association with a label;
• and
• retraining the trained machine learning model using the supplemented data sample set.