Scalable Pseudo Labelling Process for Classification

A method, comprising:

• receiving, by a computer system, an indication to update a training dataset for a machine learning algorithm;
• accessing, by the computer system in response to the indication, data in the training dataset for the machine learning algorithm, wherein the training dataset includes annotated labels on the data;
• accessing, by the computer system, a dataset comprising a plurality of data, wherein the data is labelled corresponding to a plurality of categories;
• selecting a subset of labelled data from the dataset, wherein the labels on the data in the subset of data correspond to the annotated labels on the data in the training dataset;
• selecting a portion of the data from the subset to add to the training dataset, wherein the portion of the data is selected based on the data with a given label in the portion having a measure of similarity, with respect to the data with the same given label in the training dataset, that satisfies a predetermined threshold;
• updating the training dataset by adding the portion of the data selected to the training dataset;
• and
• providing the updated training dataset to the machine learning algorithm for training of the machine learning algorithm.

A non-transitory computer-readable medium having instructions stored thereon that are executable by a computing device to perform operations, comprising:

• accessing a dataset comprising a plurality of data, wherein the data is labelled corresponding to a plurality of categories;
• selecting a first subset of data from the dataset by applying one or more clustering algorithms to the labelled data in the dataset;
• applying annotation to the labelled data in the first subset to refine the labels on the data in the first subset, wherein the annotation includes, at least in part,
• implementation of historical information for the data;
• selecting a second subset of labelled data from the dataset, wherein the labels on the data in the second subset of data correspond to the labels on the data in the first subset after annotation;
• selecting at least one item of data with a given label from the second subset to add to the first subset, wherein the at least one item of data with the given label is selected based on the at least one item of data with the given label having a ranking of similarity, with respect to the data with the same given label in the first subset, that satisfies a predetermined threshold;
• and
• adding the at least one item of data with the given label to the first subset to generate a training dataset for a machine learning algorithm.