Systems and methods for improved active learning method for model development

A method for minimizing resource expenditure during model training using user-defined constraints in sample selection, the method comprising:

• obtaining (1) one or more user-defined target parameter values for data labeling, (2) a user input indicative of a value added per unit of model performance improvement, and (3) a dataset comprising a plurality of unlabeled samples;
• selecting a first subset of the dataset, wherein the first subset comprises samples of the plurality of unlabeled samples;
• transmitting, to a remote device, a request for labeling the samples of the first subset, wherein the request comprises the samples of the first subset;
• receiving, from the remote device, a first training dataset based on the first subset, wherein the first training dataset comprises label data and the samples of the first subset, and wherein the label data indicates a classification for each sample;
• training, using the first training dataset, a machine learning model;
• generating, using the user input indicative of the value added per unit of model performance improvement, a margin curve of a relationship between resource usage and value added per unit of model performance improvement;
• determining, based on the margin curve, whether an amount of resource usage exceeds an amount of value added;
• and
• responsive to determining that the amount of value added does not exceed the amount of resource usage, selecting a second subset of the dataset, wherein a number of samples of the second subset is determined based on the margin curve.

One or more non-transitory computer readable media for minimizing resource expenditure during model training using user-defined constraints in sample selection, storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

• obtaining (1) one or more user-defined target parameter values for data labeling, (2) a user input indicative of a value added per unit of model performance improvement, and (3) a dataset comprising a plurality of unlabeled samples;
• selecting a first subset of the dataset, wherein the first subset comprises samples of the plurality of unlabeled samples;
• transmitting, to a remote device, a request for labeling the samples of the first subset, wherein the request comprises the samples of the first subset;
• receiving, from the remote device, a first training dataset based on the first subset, wherein the first training dataset comprises label data and the samples of the first subset, and wherein the label data indicates a classification for each sample;
• training, using the first training dataset, a machine learning model;
• generating, using the user input indicative of the value added per unit of model performance improvement, a margin curve of a relationship between resource usage and value added per unit of model performance improvement;
• determining, based on the margin curve, whether an amount of resource usage exceeds an amount of value added;
• and
• responsive to determining that the amount of value added does not exceed the amount of resource usage, selecting a second subset of the dataset, wherein a number of samples of the second subset is determined based on the margin curve.