Distributed hyperparameter tuning system for active machine learning

A computing device comprising:

• a processor;
• and
• a computer-readable medium operably coupled to the processor, the computer-readable medium having computer-readable instructions stored thereon that, when executed by the processor, cause the computing device to
• (A) determine a first loss value based on a converged classification matrix computed using a plurality of observation vectors, wherein the plurality of observation vectors includes a plurality of classified observation vectors and a plurality of unclassified observation vectors, wherein the converged classification matrix defines a label probability for each permissible classification value of a plurality of permissible classification values for each observation vector of the plurality of unclassified observation vectors, wherein the converged classification matrix is defined using a kernel function and a predefined kernel parameter value, wherein each observation vector of the plurality of classified observation vectors has one of the plurality of permissible classification values defined for a target variable;
• (B) assign each observation vector of the plurality of observation vectors to a cluster using a clustering algorithm based on the converged classification matrix to define a plurality of clusters;
• (C) select a predefined number of observation vectors from each cluster of the defined plurality of clusters based on an entropy value computed for each observation vector of the plurality of unclassified observation vectors, wherein the predefined number of observation vectors is based on a predefined batch size value;
• (D) update a copy of the plurality of classified observation vectors to include the observation vectors selected in (C) with a value for the target variable selected based on the label probability of each permissible classification value for each respective selected observation vector;
• (E) update a copy of the plurality of unclassified observation vectors to remove the observation vectors selected in (C);
• (F) repeat (A) with the updated copy of the plurality of classified observation vectors and the updated copy of the plurality of unclassified observation vectors;
• (G) determine an entropy loss value using the entropy value computed for each observation vector selected in (C), wherein (A) to (G) are repeated for a plurality of different values of the predefined kernel parameter value and the predefined batch size value;
• (H) determine a second loss value based on the converged classification matrix, a label matrix defined from the converged classification matrix, and a predefined weight value, wherein the label matrix has one of the plurality of permissible classification values defined for the target variable for each observation vector of the updated plurality of unclassified observation vectors;
• (I) select a second number of observation vectors from the plurality of unclassified observation vectors, wherein the second number of observation vectors is determined from the different values of the predefined batch size value based on the determined first loss value, the determined entropy loss value, and the determined second loss value;
• (J) update the plurality of classified observation vectors to include the observation vectors selected in (I) with the value for the target variable selected based on the label probability of each permissible classification value for each respective observation vector selected in (I);
• (K) update the plurality of unclassified observation vectors to remove the observation vectors selected in (I);
• (L) repeat (A) to (K) with a plurality of different values of the predefined weight value until convergence is satisfied;
• and
• (M) output the label matrix to classify each observation vector included in the plurality of unclassified observation vectors.

A method of classifying data using automatically selected hyperparameter values, the method comprising:

• (A) determining, by a computing device, a first loss value based on a converged classification matrix computed using a plurality of observation vectors, wherein the plurality of observation vectors includes a plurality of classified observation vectors and a plurality of unclassified observation vectors, wherein the converged classification matrix defines a label probability for each permissible classification value of a plurality of permissible classification values for each observation vector of the plurality of unclassified observation vectors, wherein the converged classification matrix is defined using a kernel function and a predefined kernel parameter value, wherein each observation vector of the plurality of classified observation vectors has one of the plurality of permissible classification values defined for a target variable;
• (B) assigning, by the computing device, each observation vector of the plurality of observation vectors to a cluster using a clustering algorithm based on the converged classification matrix to define a plurality of clusters;
• (C) selecting, by the computing device, a predefined number of observation vectors from each cluster of the defined plurality of clusters based on an entropy value computed for each observation vector of the plurality of unclassified observation vectors, wherein the predefined number of observation vectors is based on a predefined batch size value;
• (D) updating, by the computing device, a copy of the plurality of classified observation vectors to include the observation vectors selected in (C) with a value for the target variable selected based on the label probability of each permissible classification value for each respective selected observation vector;
• (E) updating, by the computing device, a copy of the plurality of unclassified observation vectors to remove the observation vectors selected in (C);
• (F) repeating, by the computing device, (A) with the updated copy of the plurality of classified observation vectors and the updated copy of the plurality of unclassified observation vectors;
• (G) determining, by the computing device, an entropy loss value using the entropy value computed for each observation vector selected in (C), wherein (A) to (G) are repeated for a plurality of different values of the predefined kernel parameter value and the predefined batch size value;
• (H) determining, by the computing device, a second loss value based on the converged classification matrix, a label matrix defined from the converged classification matrix, and a predefined weight value, wherein the label matrix has one of the plurality of permissible classification values defined for the target variable for each observation vector of the updated plurality of unclassified observation vectors;
• (I) selecting, by the computing device, a second number of observation vectors from the plurality of unclassified observation vectors, wherein the second number of observation vectors is determined from the different values of the predefined batch size value based on the determined first loss value, the determined entropy loss value, and the determined second loss value;
• (J) updating, by the computing device, the plurality of classified observation vectors to include the observation vectors selected in (I) with the value for the target variable selected based on the label probability of each permissible classification value for each respective observation vector selected in (I);
• (K) updating, by the computing device, the plurality of unclassified observation vectors to remove the observation vectors selected in (I);
• (L) repeating, by the computing device, (A) to (K) with a plurality of different values of the predefined weight value until convergence is satisfied;
• and
• (M) outputting, by the computing device, the label matrix to classify each observation vector included in the plurality of unclassified observation vectors.