Active learning-based data labeling service using an augmented manifest

A computer-implemented method comprising:

• receiving a plurality of objects using a reference to a data store;
• receiving an augmented manifest for the plurality of objects, each object from the plurality of objects corresponding to a separate line in the augmented manifest, by:
• receiving a plurality of files of a data type in the data store, the plurality of files associated with the plurality of objects;
• extracting metadata from the plurality of files;
• and
• generating the augmented manifest in the data store based on the metadata;
• receiving a subset of the plurality of objects to be labeled;
• executing a labeling job on the subset of the plurality of objects, the labeling job identifying a plurality of labels associated with the subset of the plurality of objects and generating a labeled subset of the plurality of objects;
• updating the augmented manifest based on the plurality of labels identified in the labeling job, each label added to a line of the augmented manifest that is associated with an object from the subset of the plurality of objects corresponding to the label, the augmented manifest to be used as input to a second labeling job;
• and
• incrementally training an active learning service model using the labeled subset of the plurality of objects, wherein the active learning service includes a validation dataset, and sends feedback on the validation dataset to a dataset service to remove particular objects from the validation dataset.

A system comprising:

• a data labeling service implemented by a second one or more electronic devices, the data labeling service including instructions that upon execution cause the data labeling service to:
• receive a plurality of objects using a reference to a data store;
• receive an augmented manifest for the plurality of objects, each object from the plurality of objects corresponding to a separate line in the augmented manifest, by:
• receiving a plurality of files of a data type from the data store, the plurality of files associated with the plurality of objects;
• extracting metadata from the plurality of files;
• and
• generating the augmented manifest in the data store based on the metadata;
• receive a subset of the plurality of objects to be labeled;
• execute a labeling job on the subset of the plurality of objects, the labeling job identifying a plurality of labels associated with the subset of the plurality of objects and generating a labeled subset of the plurality of objects;
• update the augmented manifest based on the plurality of labels identified in the labeling job, each label added to a line of the augmented manifest that is associated with an object from the subset of the plurality of objects corresponding to the label, the augmented manifest to be used as input to a second labeling job;
• and
• incrementally train an active learning service model using the labeled subset of the plurality of objects, wherein the active learning service includes a validation dataset, and sends feedback on the validation dataset to a dataset service to remove particular objects from the validation dataset.