Optimizing training data for image classification

A system for machine learning-based classification, the system comprising:

• a processor;
• and
• a non-transitory, computer-readable memory storing instructions that, when executed by the processor, cause the processor to:
• obtain training data comprising a full training data set;
• train a machine learning model with the full training data set to a first model state;
• generate respective embeddings for the data points in the full training data set with the first model state of the machine learning model;
• apply a clustering algorithm to the respective embeddings to generate a plurality of clusters of the embeddings;
• identify outlier embeddings from the plurality of clusters of the embeddings;
• generate a reduced training data set comprising the full training data set less the data points associated with the outlier embeddings, the reduced training data set including the data points associated with embeddings in the plurality of clusters;
• train the machine learning model with the reduced training data set to a second model state;
• and
• apply the second model state to one or more data sets to classify the one or more data sets.

A machine learning-based method of classifying a plurality of images, the method comprising:

• training a machine learning model with a full training data set, the full training data set comprising a plurality of paired images and classes, to generate a first model state of the machine learning model;
• generating respective embeddings for the images in the full training data set with the first model state of the machine learning model;
• applying a clustering algorithm to the respective embeddings to generate a plurality of clusters of the embeddings;
• identifying outlier embeddings from the plurality of clusters of the embeddings;
• generating a reduced training data set comprising the full training data set less the images associated with the outlier embeddings, the reduced training data set including the images associated with the embeddings in the plurality of clusters;
• training the machine learning model with the reduced training data set to a second model state;
• and
• applying the second model state to one or more unclassified images to classify the one or more unclassified images.