Methods and Devices Relating to Estimating Classifier Performance

A computer-implemented method comprising:

• using suitably programmed hardware to associate a feature variability with at least one of one or more features on a feature-by-feature basis, the one or more features characterizing a sample from multiple samples, each of which is characterized by one or more features;
• computing, with the hardware, a first probability of misclassification by a neural network classifier using the feature variability;
• and
• computing, with the hardware, a first probability of misclassification by a classifier using the feature variability, where the classifier can be used to determine a class to which a sample belongs based on the one or more features that characterize that sample.

A computer-implemented method comprising:

• using suitably programmed hardware to associate a feature variability with at least one of one or more features on a feature-by-feature basis, the one or more features characterizing a sample from multiple samples, each of which is characterized by one or more features;
• and
• computing, with the hardware, a first probability of misclassification by a first classifier using the feature variability, where the first classifier can be used to determine a class to which a sample belongs based on the one or more features that characterize that sample;
• where the computing, with the hardware, a first probability of misclassification by a first classifier includes using a linear approximation of the first classifier prior to compute the first probability of misclassification.

A computer-implemented method for estimating performance of a classifier comprising:

• using suitably programmed hardware to associate a feature variability with at least one of one or more features that characterize a sample from multiple samples, each of which is characterized by one or more features;
• using a first classifier to compute, with the hardware, one or more first scores for one or more first samples in a subset of samples from the multiple samples, where the first classifier can be used to determine a class to which a sample belongs based on the one or more features that characterize that sample;
• using the hardware to associate a first score variability with at least one of the one or more first scores using the feature variability;
• computing, with the hardware, a first probability of misclassification by the first classifier using one or more of the one or more first scores and the first score variability;
• using a second classifier to compute, with the hardware, one or more second scores for one or more second samples in a subset of samples from the multiple samples, where the second classifier can be used to determine a class to which a sample belongs based on the one or more features that characterize that sample;
• using the hardware to associate a second score variability with at least one of the one or more second scores using the feature variability;
• computing, with the hardware, a second probability of misclassification by the second classifier using one or more of the one or more second scores and the second score variability;
• and
• determining, with the hardware, a final probability of misclassification from a group of probabilities of misclassification that includes the first and second probabilities of misclassification.