Automatic thresholding for classification models

A computer-implemented method for selecting a classification threshold value usable to control a classification decision rendered by a trained machine learning model, comprising:

• executing the trained machine learning model against an input data set;
• for each of a plurality of predetermined classification threshold values usable to control the classification decision rendered by the trained machine learning model:
• determining at least one accuracy metric of the trained machine learning model based on said executing;
• modeling a precision of the trained machine learning model based at least on the at least one accuracy metric to generate a modeled precision value;
• modeling a recall of the trained machine learning model based at least one the at least one accuracy metric to generate a modeled recall value;
• determining a probability that the modeled precision value has a first predetermined relationship with a target precision value;
• determining that the probability has a second predetermined relationship with a predetermined confidence level;
• and
• responsive to determining that the probability has the second predetermined relationship with the predetermined confidence level, adding the predetermined classification threshold value to a set of candidate classification threshold values;
• and
• selecting the classification threshold value from among the set of candidate classification threshold values by selecting the candidate classification threshold value associated with the largest modeled recall value.

A computer-readable storage medium having program instructions recorded thereon that, when executed by at least one processor, perform a method for selecting a classification threshold value usable to control a classification decision rendered by a trained machine learning model, the method comprising:

• executing the trained machine learning model against an input data set;
• for each of a plurality of predetermined classification threshold values usable to control the classification decision rendered by the trained machine learning model:
• determining at least one accuracy metric of the trained machine learning model based on said executing;
• modeling a first evaluation metric of the trained machine learning model based at least on the at least one accuracy metric to generate a first modeled evaluation metric value;
• modeling a second evaluation metric of the trained machine learning model based at least one the at least one accuracy metric to generate a second modeled evaluation metric value;
• determining a probability that the first modeled evaluation metric value has a first predetermined relationship with a target evaluation metric value;
• determining that the probability has a second predetermined relationship with a predetermined confidence level;
• and
• responsive to determining that the probability has the second predetermined relationship with the predetermined confidence level, adding the predetermined classification threshold value to a set of candidate classification threshold values;
• and
• selecting the classification threshold value from among the set of candidate classification threshold values by selecting the candidate classification threshold value associated with the largest second modeled evaluation metric value.