System and method for automated content annotation workflow

A method for improving the volume and quality of training data for an artificial intelligence (AI) system, the method comprising:

• registering, by use of a data processor, a plurality of labelers to which annotation tasks are assigned;
• populating, by use of the data processor, a labeling queue with content data to be annotated;
• assigning, by use of the data processor, annotation tasks from the labeling queue to the plurality of labelers and prompt the plurality of labelers to apply annotations to objects identified in the content data;
• enabling, by use of the data processor, a plurality of labelers in an annotation review queue to modify or delete annotations applied by prior labelers;
• using a trained machine learning model to produce predictions of potential objects, features, or patterns of interest in the content data being analyzed by the plurality of labelers;
• generating, by use of the data processor, predicted annotations from the predictions produced by the trained machine learning model;
• enabling the plurality of labelers to review and modify the predicted annotations;
• evaluating, by use of the data processor, a level of performance of the plurality of labelers in applying the annotations;
• and
• exporting annotations having a corresponding confidence level at or above a pre-defined threshold to a machine learning training system using programmatic data transfers.

A non-transitory machine-useable storage medium embodying instructions for improving the volume and quality of training data for an artificial intelligence (AI) system, the instructions when executed by a machine, cause the machine to:

• register, by use of a data processor, a plurality of labelers to which annotation tasks are assigned;
• populate, by use of the data processor, a labeling queue with content data to be annotated;
• assign, by use of the data processor, annotation tasks from the labeling queue to the plurality of labelers and prompt the plurality of labelers to apply annotations to objects identified in the content data;
• enable, by use of the data processor, a plurality of labelers in an annotation review queue to modify or delete annotations applied by prior labelers;
• use a trained machine learning model to produce predictions of potential objects, features, or patterns of interest in the content data being analyzed by the plurality of labelers;
• generate, by use of the data processor, predicted annotations from the predictions produced by the trained machine learning model;
• enable the plurality of labelers to review and modify the predicted annotations;
• evaluate, by use of the data processor, a level of performance of the plurality of labelers in applying the annotations;
• and
• export annotations having a corresponding confidence level at or above a pre-defined threshold to a machine learning training system using programmatic data transfers.