Methods and apparatus for assisted data review for active learning cycles

A non-transitory computer readable storage medium comprising instructions to cause programmable circuitry to at least:

• determine a first training loss associated with a first data point of training data for training a machine learning model;
• determine a second training loss associated with a second data point of the training data;
• rank the training data based on aggregate statistics of the first and second training losses;
• select, based on the rank, the first data point for annotation;
• and
• modify an existing label of the first data point based on the annotation.

A method comprising:

• determining a first training loss associated with a first data point of training data for training a machine learning model;
• determining a second training loss associated with a second data point of the training data;
• ranking the training data based on aggregate statistics of the first and second training losses;
• selecting, based on the ranking, the first data point for annotation;
• and
• modifying an existing label of the first data point based on the annotation.