Computer-implemented methods and systems for optimal linear classification systems

A computer implemented method of multiclass linear classification, comprising:

• receiving M sets of d-dimensional feature vectors that have been extracted from a common digital data source;
• and
• producing an ensemble of M−1 linear classifiers for each of the M pattern classes by transforming M sets of d-dimensional feature vectors, where the feature vectors in an ensemble of M−1 linear classifiers for a given pattern class have the class membership statistic +1 and the feature vectors in all of the other pattern classes have the class membership statistic −1, into M−1 data-driven likelihood ratio tests, each of which is an indicator function for a given pattern class that is based on a dual locus of likelihoods and principal eigenaxis components, formed by a locus of weighted extreme points, where each weight specifies a class membership statistic and a conditional density for an extreme point, which is located in either an overlapping region or a tail region between two data distributions, and each weight determines the magnitude and the total allowed eigenenergy of an extreme vector, such that each dual locus of likelihoods and principal eigenaxis components is the basis of an optimal linear classification system that exhibits the highest accuracy and achieves Bayes' error rate for feature vectors drawn from statistical distributions that that have similar covariance functions and constant or unchanging means and covariance statistics, where each optimal linear classification system is an indicator function for a given pattern class;
• according to a system of fundamental, data-driven, vector-based locus equations of binary classification for a linear classification system in statistical equilibrium that determines fundamental equations of statistical equilibrium along with fundamental equations of minimization of eigenenergy and Bayes' risk, which are satisfied by a data-driven likelihood ratio test that contains Bayes' likelihood ratio and delineates an optimal linear decision boundary;
• and
• forming linear combinations of the M−1 linear classifiers for each of the M pattern classes to produce M ensembles of M−1 linear classification systems;
• and
• forming linear combinations of the M ensembles to produce an M-class linear classification system;
• and
• identifying class memberships of unknown feature vectors according to the output of the ensemble of M−1 linear classifiers.

A computer implemented method of fusing M-class linear classification systems using feature vectors that have been extracted from two different types of data sources, comprising:

• receiving M sets of d-dimensional feature vectors and M sets of n-dimensional feature vectors that have been extracted from two different sources of digital data;
• and
• producing two ensembles of M−1 linear classifiers for each of the M pattern classes by transforming the M sets of d-dimensional feature vectors and the M sets of n-dimensional feature vectors, where the feature vectors in an ensemble of M−1 quadratic classifiers for a given pattern class have the class membership statistic +1 and the feature vectors in all of the other pattern classes have the class membership statistic −1, into two ensembles of M−1 data-driven likelihood ratio tests, where each data-driven likelihood ratio test is an indicator function for a given pattern class that is based on a dual locus of likelihoods and principal eigenaxis components, formed by a locus of weighted extreme points, where each weight specifies a class membership statistic and a conditional density for an extreme point, which is located in either an overlapping region or a tail region between two data distributions, and each weight determines the magnitude and the total allowed eigenenergy of an extreme vector, such that each dual locus of likelihoods and principal eigenaxis components is the basis of an optimal linear classification system that exhibits the highest accuracy and achieves Bayes' error rate for feature vectors drawn from statistical distributions that have similar covariance functions and constant or unchanging means and covariance statistics, where each optimal linear classification system is an indicator function for a given pattern class;
• according to a system of fundamental, data-driven, vector-based locus equations of binary classification for a linear classification system in statistical equilibrium that determines fundamental equations of statistical equilibrium along with fundamental equations of minimization of eigenenergy and Bayes' risk, which are satisfied by a data-driven likelihood ratio test that contains Bayes' likelihood ratio and delineates an optimal linear decision boundary;
• and
• forming linear combinations of both ensembles of M−1 linear classifiers for each of the M pattern classes to produce two sets of M ensembles of M−1 linear classification systems;
• and
• forming linear combinations of the two sets of M ensembles of M−1 linear classification systems for each of the M pattern classes to produce an M-class linear classification system;
• and
• identifying class memberships of unknown feature vectors according to the output of the fused ensembles of M−1 linear classifiers.

A computer implemented method of using linear classification systems to measure data distribution overlap and Bayes' error rate for two given sets of feature vectors, comprising:

• transforming two sets of feature vectors that are identified as members of two predefined classes into a practical statistical gauge, which accurately measures the data distribution overlap and the Bayes' error rate for the two given sets of feature vectors, that consists of a data-driven likelihood ratio test that is based on a dual locus of likelihoods and principal eigenaxis components, formed by a locus of weighted extreme points, where each weight specifies a class membership statistic and a conditional density for an extreme point, which is located in either an overlapping region or a tail region between two data distributions, and each weight determines the magnitude and the total allowed eigenenergy of an extreme vector, such that the dual locus of likelihoods and principal eigenaxis components is the basis of an optimal linear classification system that exhibits the highest accuracy and achieves Bayes' error rate for feature vectors drawn from statistical distributions that have similar covariance functions and constant or unchanging mean and covariance statistics;
• according to a system of fundamental, data-driven, vector-based locus equations of binary classification for a linear classification system in statistical equilibrium that determines fundamental equations of statistical equilibrium along with fundamental equations of minimization of eigenenergy and Bayes' risk, which are satisfied by a data-driven likelihood ratio test that contains Bayes' likelihood ratio and delineates an optimal linear decision boundary;
• and
• using the linear classification system to identify the class memberships of a collection of unknown feature vectors according to the output of the optimal linear classification system, where each unknown feature vector is identified as a member of one of the two predefined classes, and
• comparing the known class memberships to the predicted class memberships;
• and
• determining the error rate for each pattern class based on the frequency of incorrect predictions for each pattern class;
• and
• determining the data distribution overlap and the Bayes' error rate based on the error rates of the collection of unknown feature vectors.

A computer implemented method of using optimal linear classification systems to identify homogeneous data distributions, comprising:

• transforming two sets of feature vectors that are identified as members of two predefined classes into a practical statistical gauge, which accurately measures the data distribution overlap and the Bayes' error rate for two given sets of feature vectors that are drawn from homogenous data distributions, that consists of a data-driven likelihood ratio test that is based on a dual locus of likelihoods and principal eigenaxis components, formed by a locus of weighted extreme points, where each weight specifies a class membership statistic and a conditional density for an extreme point, which is located in either an overlapping region or a tail region between two data distributions, and each weight determines the magnitude and the total allowed eigenenergy of an extreme vector, such that the dual locus of likelihoods and principal eigenaxis components is the basis of an optimal linear classification system that exhibits the highest accuracy and achieves Bayes' error rate of 50% for feature vectors drawn from homogeneous data distributions, where all of the feature vectors drawn from homogenous data distributions are extreme vectors;
• according to a system of fundamental, data-driven, vector-based locus equations of binary classification for a linear classification system in statistical equilibrium that determines fundamental equations of statistical equilibrium along with fundamental equations of minimization of eigenenergy and Bayes' risk:
• which are satisfied by a data-driven likelihood ratio test that contains Bayes' likelihood ratio and delineates an optimal linear decision boundary;
• and
• using the optimal linear classification system to identify the class memberships of a collection of unknown feature vectors according to the output of the optimal linear classification system, where each unknown feature vector is identified as a member of one of the two predefined classes;
• and
• comparing the known class memberships to the predicted class memberships;
• and
• determining the error rate for each pattern class based on the frequency of incorrect predictions for each pattern class;
• and
• determining the data distribution overlap and the Bayes' error rate based on the number of extreme points and the error rates of the collection of unknown feature vectors;
• and
• determining if the two sets of features vectors are drawn from similar statistical distributions based on the data distribution overlap and the Bayes' error rate.