Methods for using feature vectors and machine learning algorithms to determine discriminant functions of minimum risk quadratic classification systems

A computer-implemented method of using feature vectors and machine learning algorithms to determine a fused discriminant function of a fused minimum risk quadratic classification system that classifies two types of said feature vectors into two classes, wherein said types of said feature vectors have different numbers of vector components, and using said fused discriminant function of said fused minimum risk quadratic classification system to classify unknown feature vectors related to said two types of said feature vectors, said method comprising:

• receiving an N×d data set of feature vectors within a computer system, wherein N is a number of feature vectors, d is a number of vector components in each feature vector, and each one of said N feature vectors is labeled with information that identifies which of two classes each one of said N feature vectors belongs to, and wherein each said feature vector is defined by a d-dimensional vector of numerical features, wherein said numerical features are extracted from digital signals;
• receiving an N×p different data set of different feature vectors within said computer system, wherein N is a number of different feature vectors, p is a number of vector components in each different feature vector, and each one of said N different feature vectors is labeled with information that identifies which of said two classes each one of said N different feature vectors belongs to, and wherein each said different feature vector is defined by a p-dimensional vector of numerical features, wherein said numerical features are extracted from digital signals;
• receiving within said computer system unknown feature vectors related to said data set and unknown different feature vectors related to said different data set;
• determining a kernel matrix using said data set, said determination of said kernel matrix being performed by using processors of said computer system to calculate a matrix of all possible inner products of signed reproducing kernels of said N feature vectors, wherein a reproducing kernel of a feature vector replaces said feature vector with a curve that contains first and second degree vector components, and wherein each one of said reproducing kernels of said N feature vectors has a sign of +1 or −1 that identifies which of said two classes each one of said N feature vectors belongs to, and using said processors of said computer system to calculate a regularized kernel matrix from said kernel matrix;
• determining a different kernel matrix using said different data set, said determination of said different kernel matrix being performed by using saki processors of said computer system to calculate a matrix of all possible inner products of signed reproducing kernels of said N different feature vectors, wherein a reproducing kernel of a different feature vector replaces said different feature vector with a curve that contains first and second degree vector components, and wherein each one of said reproducing kernels of said N different feature vectors has a sign of +1 or −1 that identifies which of said two classes each one of said N different feature vectors belongs to, and using said processors of said computer system to calculate a regularized different kernel matrix from said different kernel matrix;
• determining a discriminant function of a minimum risk quadratic classification system using said regularized kernel matrix and said data set, said determination of said discriminant function of said minimum risk quadratic classification system comprising the steps of:
• determining scale factors of a geometric locus of signed and scaled reproducing kernels of extreme points using said regularized kernel matrix, wherein said extreme points are located within overlapping regions or near tail regions of distributions of said N feature vectors, said determination of said scale factors being performed by using said processors of said computer system to determine a solution of a dual optimization problem, wherein said scale factors and said geometric locus satisfy a system of fundamental locus equations of binary classification, subject to geometric and statistical conditions for a minimum risk quadratic classification system in statistical equilibrium, and wherein said scale factors determine conditional densities for said extreme points and also determine critical minimum eigenenergies exhibited by scaled extreme vectors on said geometric locus, wherein said critical minimum eigenenergies determine conditional probabilities of said extreme points and also determine corresponding counter risks and risks of a minimum risk quadratic classification system, wherein said counter risks are associated with right decisions and said risks are associated with wrong decisions of said minimum risk quadratic classification system, and wherein said geometric locus determines the principal eigenaxis of the decision boundary of said minimum risk quadratic classification system, wherein said principal eigenaxis exhibits symmetrical dimensions and density, wherein said conditional probabilities and said critical minimum eigenenergies exhibited by said minimum risk quadratic classification system are symmetrically concentrated within said principal eigenaxis, and wherein counteracting and opposing components of said critical minimum eigenenergies exhibited by corresponding components of said scaled extreme vectors on said geometric locus together with corresponding counter risks and risks exhibited by said minimum risk quadratic classification system are symmetrically balanced with each other about the geometric center of said principal eigenaxis, wherein the center of total allowed eigenenergy and minimum expected risk of said minimum risk quadratic classification system is located at the geometric center of said geometric locus, and wherein said geometric locus determines a primal representation of a dual locus of likelihood components and principal eigenaxis components, wherein said likelihood components and said principal eigenaxis components are symmetrically distributed over either side of the axis of said dual locus, wherein a statistical fulcrum is placed directly under the center of said dual locus, and wherein said likelihood components of said dual locus determine conditional likelihoods for said extreme points, and wherein said principal eigenaxis components of said dual locus determine an intrinsic coordinate system of geometric loci of a quadratic decision boundary and corresponding decision borders that jointly partition the decision space of said minimum risk quadratic classification system into symmetrical decision regions;
• determining said extreme vectors on said geometric locus using the vector of said scale factors, said determination of said extreme vectors being performed by using said processors of said computer system to identify said scale factors that exceed zero by a small threshold, and using said processors of said computer system to determine a sign vector of signs associated with said extreme vectors using said data set, and compute the average sign using said sign vector;
• determining a locus of risk for said minimum risk quadratic classification system using said reproducing kernels of said extreme vectors and said signed reproducing kernels of said N feature vectors and said vector of scale factors, said determination of said locus of risk being performed by using said processors of said computer system to calculate a matrix of inner products between said signed reproducing kernels of said N feature vectors and said reproducing kernels of said extreme vectors, and multiply said matrix by said vector of scale factors, and compute the average risk for said minimum risk quadratic classification system using said locus of risk;
• determining a discriminant locus for said minimum risk quadratic classification system using said geometric locus, said determination of said discriminant locus being performed by using said processors of said computer system to calculate a matrix of inner products between said signed reproducing kernels of said N feature vectors and said reproducing kernels of said unknown feature vectors, and multiply said matrix by said vector of scale factors;
• determining the discriminant function of said minimum risk quadratic classification system, using said average risk and said average sign and said discriminant locus, said determination of said discriminant function of said minimum risk quadratic classification system being performed by using said processors of said computer system to subtract said average risk from sum of said discriminant locus and said average sign, wherein said discriminant function of said minimum risk quadratic classification system satisfies said system of fundamental locus equations of binary classification, and wherein said discriminant function of said minimum risk quadratic classification system determines likely locations of said N feature vectors and also determines said geometric loci of said quadratic decision boundary and said corresponding decision borders that jointly partition said extreme points into said symmetrical decision regions, wherein said symmetrical decision regions span said overlapping regions or said tail regions of said distributions of said N feature vectors, and wherein said discriminant function of said minimum risk quadratic classification system satisfies said quadratic decision boundary in terms of a critical minimum eigenenergy and said minimum expected risk, wherein said counteracting and opposing components of said critical minimum eigenenergies exhibited by said corresponding components of said scaled extreme vectors on said geometric locus associated with said corresponding counter risks and risks exhibited by said minimum risk quadratic classification system are symmetrically distributed over said axis of said dual locus, on equal sides of said statistical fulcrum located at said geometric center of said dual locus, wherein said counteracting and opposing components of said critical minimum eigenenergies together with said corresponding counter risks and risks exhibited by said minimum risk quadratic classification system are symmetrically balanced with each other about said geometric center of said dual locus, and wherein said statistical fulcrum is located at said center of said total allowed eigenenergy and said minimum expected risk of said minimum risk quadratic classification system, wherein said minimum risk quadratic classification system satisfies a state of statistical equilibrium, wherein said total allowed eigenenergy and said expected risk of said minimum risk quadratic classification system are minimized, and wherein said minimum risk quadratic classification system exhibits the minimum probability of error for classifying said N feature vectors that belong to said two classes and said unknown feature vectors related to said data set;
• determining a different discriminant function of a different minimum risk quadratic classification system using said regularized different kernel matrix and said different data set, said determination of said different discriminant function of said different minimum risk quadratic classification system being performed by using said processors of said computer system to perform said steps of determining said discriminant function of said minimum risk quadratic classification system, wherein said different minimum risk quadratic classification system exhibits the minimum probability of error for classifying said N different feature vectors that belong to said two classes and said unknown different feature vectors related to said different data set;
• determining a fused discriminant function of a fused minimum risk quadratic classification system using said discriminant function of said minimum risk quadratic classification system and said different discriminant function of said different minimum risk quadratic classification system, said determination of said fused discriminant function of said fused minimum risk quadratic classification system being performed by using said processors of said computer system to sum said discriminant function of said minimum risk quadratic classification system and said different discriminant function of said different minimum risk quadratic classification system;
• and
• determining which of said two classes said unknown feature vectors and said unknown different feature vectors belong to using said fused discriminant function of said fused minimum risk quadratic classification system, said determination of said classes of said unknown feature vectors and said unknown different feature vectors being performed by using said processors of said computer system to apply said fused discriminant function of said fused minimum risk quadratic classification system to said unknown feature vectors and said unknown different feature vectors, wherein said fused discriminant function determines likely locations of said unknown feature vectors and said unknown different feature vectors and identifies said decision regions related to said two classes that said unknown feature vectors and said unknown different feature vectors are located within, wherein said fused discriminant function recognizes said classes of said unknown feature vectors and said unknown different feature vectors, and wherein said fused minimum risk quadratic classification system decides which of said two classes said unknown feature vectors and said unknown different feature vectors belong to and thereby classifies said unknown feature vectors and said unknown different feature vectors.

A computer-implemented method of using feature vectors and machine learning algorithms to determine a discriminant function of an M-class minimum risk quadratic classification system that classifies said feature vectors into M classes and using said discriminant function of said M-class minimum risk quadratic classification system to classify unknown feature vectors related to said feature vectors, said method comprising:

• receiving M N×d data sets of feature vectors within a computer system, wherein M is a number of classes, N is a number of feature vectors in each one of said M data sets, d is a number of vector components in each feature vector, and each one of said N feature vectors in each one of said M data sets belongs to the same class and is labeled with information that identifies said class, and wherein each said feature vector is defined by a d-dimensional vector of numerical features, wherein said numerical features are extracted from digital signals;
• receiving within said computer system unknown feature vectors related to said M data sets;
• determining M ensembles of M−1 discriminant functions of M−1 minimum risk quadratic classification systems using said M data sets, wherein the determination of each one of said M ensembles comprises the steps of:
• determining M−1 kernel matrices for a class of feature vectors using said M data sets, said determination of said M-1 kernel matrices being performed by using processors of said computer system to calculate M−1 matrices, wherein each matrix contains all possible inner products of signed reproducing kernels of feature vectors that belong to said class and one of the other M-1 classes, wherein a reproducing kernel of a feature vector replaces said feature vector with a curve that contains first and second degree vector components, and wherein said N feature vectors that belong to said class have the sign +1, and said N feature vectors that belong to said other class have the sign −1, and wherein said M−1 matrices account for all of the other said M−1 classes, and calculating M−1 regularized kernel matrices from said M−1 kernel matrices;
• determining M−1 discriminant functions of M−1 minimum risk quadratic classification systems using said M−1 regularized kernel matrices and said M data sets, wherein the determination of each one of said M−1 discriminant functions of M−1 minimum risk quadratic classification systems further comprises the steps of:
• determining scale factors of a geometric locus of signed and scaled reproducing kernels of extreme points using one of said regularized kernel matrices, wherein said extreme points are located within overlapping regions or near tail regions of distributions of feature vectors that belong to said class and one of the other said M−1 classes, said determination of said scale factors being performed by using said processors of said computer system to determine a solution of a dual optimization problem, wherein said scale factors and said geometric locus satisfy a system of fundamental locus equations of binary classification, subject to geometric and statistical conditions for a minimum risk quadratic classification system in statistical equilibrium, and wherein said scale factors determine conditional densities for said extreme points and also determine critical minimum eigenenergies exhibited by scaled extreme vectors on said geometric locus, wherein said critical minimum eigenenergies determine conditional probabilities of said extreme points and also determine corresponding counter risks and risks of a minimum risk quadratic classification system, wherein said counter risks are associated with right decisions and said risks are associated with wrong decisions of said minimum risk quadratic classification system, and wherein said geometric locus determines the principal eigenaxis of the decision boundary of said minimum risk quadratic classification system, wherein said principal eigenaxis exhibits symmetrical dimensions and density, wherein said conditional probabilities and said critical minimum eigenenergies exhibited by said minimum risk quadratic classification system are symmetrically concentrated within said principal eigenaxis, and wherein counteracting and opposing components of said critical minimum eigenenergies exhibited by corresponding components of said scaled extreme vectors on said geometric locus together with corresponding counter risks and risks exhibited by said minimum risk quadratic classification system are symmetrically balanced with each other about the geometric center of said principal eigenaxis, wherein the center of total allowed eigenenergy and minimum expected risk of said minimum risk quadratic classification system is located at the geometric center of said geometric locus, and wherein said geometric locus determines a primal representation of a dual locus of likelihood components and principal eigenaxis components, wherein said likelihood components and said principal eigenaxis components are symmetrically distributed over either side of the axis of said dual locus, wherein a statistical fulcrum is placed directly under the center of said dual locus, and wherein said likelihood components of said dual locus determine conditional likelihoods for said extreme points, and wherein said principal eigenaxis components of said dual locus determine an intrinsic coordinate system of geometric loci of a quadratic decision boundary and corresponding decision borders that jointly partition the decision space of said minimum risk quadratic classification system into symmetrical decision regions;
• determining said extreme vectors on said geometric locus using the vector of said scale factors, said determination of said extreme vectors being performed by using said processors of said computer system to identify said scale factors that exceed zero by a small threshold, and using said processors of said computer system to determine a sign vector of signs associated with said extreme vectors using data set of said class and data set of said other class, and compute the average sign using said sign vector;
• determining a locus of risk for said minimum risk quadratic classification system using said reproducing kernels of said extreme vectors and said signed reproducing kernels of said N feature vectors and said vector of scale factors, said determination of said locus of risk being performed by using said processors of said computer system to calculate a matrix of inner products between said signed reproducing kernels of said N feature vectors and said reproducing kernels of said extreme vectors, and multiply said matrix by said vector of scale factors, and compute the average risk for said minimum risk quadratic classification system using said locus of risk;
• determining a discriminant locus for said minimum risk quadratic classification system using said geometric locus, said determination of said discriminant locus being performed by using said processors of said computer system to calculate a matrix of inner products between said signed reproducing kernels of said feature vectors that belong to said class and said other class and said reproducing kernels of said unknown feature vectors, and multiply said matrix by said vector of scale factors;
• determining the discriminant function of said minimum risk quadratic classification system, using said average risk and said average sign and said discriminant locus, said determination of said discriminant function of said minimum risk quadratic classification system being performed by using said processors of said computer system to subtract said average risk from sum of said discriminant locus and said average sign, wherein said discriminant function of said minimum risk quadratic classification system satisfies said system of fundamental locus equations of binary classification, and wherein said discriminant function of said minimum risk quadratic classification system determines likely locations of said N feature vectors from said class and said N feature vectors from said other class and also determines said geometric loci of said quadratic decision boundary and said corresponding decision borders that jointly partition said extreme points into said symmetrical decision regions, wherein said symmetrical decision regions span said overlapping regions or said tail regions of said distributions of said N feature vectors that belong to said class and said N feature vectors that belong to said other class, and wherein said discriminant function of said minimum risk quadratic classification system satisfies said quadratic decision boundary in terms of a critical minimum eigenenergy and said minimum expected risk, wherein said counteracting and opposing components of said critical minimum eigenenergies exhibited by said corresponding components of said scaled extreme vectors on said geometric locus associated with said corresponding counter risks and risks exhibited by said minimum risk quadratic classification system are symmetrically distributed over said axis of said dual locus, on equal sides of said statistical fulcrum located at said geometric center of said dual locus, wherein said counteracting and opposing components of said critical minimum eigenenergies together with said corresponding counter risks and risks exhibited by said minimum risk quadratic classification system are symmetrically balanced with each other about said geometric center of said dual locus, and wherein said statistical fulcrum is located at said center of said total allowed eigenenergy and said minimum expected risk of said minimum risk quadratic classification system, wherein said minimum risk quadratic classification system satisfies a state of statistical equilibrium, wherein said total allowed eigenenergy and said expected risk of said minimum risk quadratic classification system are minimized, and wherein said minimum risk quadratic classification system exhibits the minimum probability of error for classifying said N feature vectors that belong to said class and said N feature vectors that belong to said other class and said unknown feature vectors related to said data set of said class and said data set of said other class;
• determining a discriminant function of an M-class minimum risk quadratic classification system using said M ensembles of M−1 discriminant functions of M−1 minimum risk quadratic classification systems, said determination of said discriminant function of said M-class minimum risk quadratic classification system being performed by using said processors of said computer system to sum said M ensembles of M−1 discriminant functions of M−1 minimum risk quadratic classification systems;
• and determining which of said M classes said unknown feature vectors belong to using said discriminant function of said M-class minimum risk quadratic classification system, said determination of said classes of said unknown feature vectors being performed by using said processors of said computer system to apply said discriminant function of said M- class minimum risk quadratic classification system to said unknown feature vectors, wherein said discriminant function determines likely locations of said unknown feature vectors and identifies said decision regions related to said M classes that said unknown feature vectors are located within, wherein said discriminant function recognizes said classes of said unknown feature vectors, and wherein said M-class minimum risk quadratic classification system decides which of said M classes said unknown feature vectors belong to and thereby classifies said unknown feature vectors.

A computer-implemented method of using feature vectors and machine learning algorithms to determine a fused discriminant function of a fused M-class minimum risk quadratic classification system that classifies two types of said feature vectors into M classes, wherein said types of said feature vectors have different numbers of vector components, and using said fused discriminant function of said fused M-class minimum risk quadratic classification system to classify unknown feature vectors related to said two types of said feature vectors, said method comprising:

• receiving M N×d data sets of feature vectors within a computer system, wherein M is a number of classes, N is a number of feature vectors in each one of said M data sets, d is a number of vector components in each feature vector, and each one of said N feature vectors in each one of said M data sets belongs to the same class and is labeled with information that identifies said class, and wherein each said feature vector is defined by a d-dimensional vector of numerical features, wherein said numerical features are extracted from digital signals;
• receiving M N×p different data sets of different feature vectors within said computer system, wherein M is said number of said classes, N is a number of different feature vectors in each one of said M different data sets, p is a number of vector components in each different feature vector, and each one of said N different feature vectors in each one of said M different data sets belongs to the same class and is labeled with information that identifies said class, and wherein each said different feature vector is defined by a p-dimensional vector of numerical features, wherein said numerical features are extracted from digital signals;
• receiving within said computer system unknown feature vectors related to said M data sets and unknown different feature vectors related to said M different data sets;
• determining M ensembles of M−1 discriminant functions of M−1 minimum risk quadratic classification systems using said M data sets, wherein the determination of each one of said M ensembles comprises the steps of:
• determining M−1 kernel matrices for a class of feature vectors using said M data sets, said determination of said M−1 kernel matrices being performed by using processors of said computer system to calculate M−1 matrices, wherein each matrix contains all possible inner products of signed reproducing kernels of feature vectors that belong to said class and one of the other M−1 classes, wherein a reproducing kernel of a feature vector replaces said feature vector with a curve that contains first and second degree vector components, and wherein said N feature vectors that belong to said class have the sign +1, and said N feature vectors that belong to said other class have the sign −1, and said M−1 matrices account for all of the other said M−1 classes, and calculating M−1 regularized kernel matrices from said M−1 kernel matrices;
• determining M−1 discriminant functions of M−1 minimum risk quadratic classification systems using said M−1 regularized kernel matrices and said M data sets, wherein the determination of each one of said M−1 discriminant functions of M−1 minimum risk quadratic classification systems further comprises the steps of:
• determining scale factors of a geometric locus of signed and scaled reproducing kernels of extreme points using one of said regularized kernel matrices, wherein said extreme points are located within overlapping regions or near tail regions of distributions of feature vectors that belong to said class and one of the other said M−1 classes, said determination of said scale factors being performed by using said processors of said computer system to determine a solution of a dual optimization problem, wherein said scale factors and said geometric locus satisfy a system of fundamental locus equations of binary classification, subject to geometric and statistical conditions for a minimum risk quadratic classification system in statistical equilibrium, and wherein said scale factors determine conditional densities for said extreme points and also determine critical minimum eigenenergies exhibited by scaled extreme vectors on said geometric locus, wherein said critical minimum eigenenergies determine conditional probabilities of said extreme points and also determine corresponding counter risks and risks of a minimum risk quadratic classification system, wherein said counter risks are associated with right decisions and said risks are associated with wrong decisions of said minimum risk quadratic classification system, and wherein said geometric locus determines the principal eigenaxis of the decision boundary of said minimum risk quadratic classification system, wherein said principal eigenaxis exhibits symmetrical dimensions and density, wherein said conditional probabilities and said critical minimum eigenenergies exhibited by said minimum risk quadratic classification system are symmetrically concentrated within said principal eigenaxis, and wherein counteracting and opposing components of said critical minimum eigenenergies exhibited by corresponding components of said scaled extreme vectors on said geometric locus together with corresponding counter risks and risks exhibited by said minimum risk quadratic classification system are symmetrically balanced with each other about the geometric center of said principal eigenaxis, wherein the center of total allowed eigenenergy and minimum expected risk of said minimum risk quadratic classification system is located at the geometric center of said geometric locus, and wherein said geometric locus determines a primal representation of a dual locus of likelihood components and principal eigenaxis components, wherein said likelihood components and said principal eigenaxis components are symmetrically distributed over either side of the axis of said dual locus, wherein a statistical fulcrum is placed directly under the center of said dual locus, and wherein said likelihood components of said dual locus determine conditional likelihoods for said extreme points, and wherein said principal eigenaxis components of said dual locus determine an intrinsic coordinate system of geometric loci of a quadratic decision boundary and corresponding decision borders that jointly partition the decision space of said minimum risk quadratic classification system into symmetrical decision regions;
• determining said extreme vectors on said geometric locus using the vector of said scale factors, said determination of said extreme vectors being performed by using said processors of said computer system to identify said scale factors that exceed zero by a small threshold, and using said processors of said computer system to determine a sign vector of signs associated with said extreme vectors using data set of said class and data set of said other class, and compute the average sign using said sign vector;
• determining a locus of risk for said minimum risk quadratic classification system using said reproducing kernels of said extreme vectors and said signed reproducing kernels of said N feature vectors and said vector of scale factors, said determination of said locus of risk being performed by using said processors of said computer system to calculate a matrix of inner products between said signed reproducing kernels of said N feature vectors and said reproducing kernels of said extreme vectors, and multiply said matrix by said vector of scale factors, and compute the average risk for said minimum risk quadratic classification system using said locus of risk;
• determining a discriminant locus for said minimum risk quadratic classification system using said geometric locus, said determination of said discriminant locus being performed by using said processors of said computer system to calculate a matrix of inner products between said signed reproducing kernels of said feature vectors that belong to said class and said other class and said reproducing kernels of said unknown feature vectors, and multiply said matrix by said vector of scale factors;
• determining the discriminant function of said minimum risk quadratic classification system, using said locus of average risk and said average sign and said discriminant locus, said determination of said discriminant function of said minimum risk quadratic classification system being performed by using said processors of said computer system to subtract said average risk from sum of said discriminant locus and said average sign, wherein said discriminant function of said minimum risk quadratic classification system satisfies said system of fundamental locus equations of binary classification, and wherein said discriminant function of said minimum risk quadratic classification system determines likely locations of said N feature vectors from said class and said N feature vectors from said other class and also determines said geometric loci of said quadratic decision boundary and said corresponding decision borders that jointly partition said extreme points into said symmetrical decision regions, wherein said symmetrical decision regions span said overlapping regions or said tail regions of said distributions of said N feature vectors that belong to said class and said N feature vectors that belong to said other class, and wherein said discriminant function of said minimum risk quadratic classification system satisfies said quadratic decision boundary in terms of a critical minimum eigenenergy and said minimum expected risk, wherein said counteracting and opposing components of said critical minimum eigenenergies exhibited by said corresponding components of said scaled extreme vectors on said geometric locus associated with said corresponding counter risks and risks exhibited by said minimum risk quadratic classification system are symmetrically distributed over said axis of said dual locus, on equal sides of said statistical fulcrum located at said geometric center of said dual locus, wherein said counteracting and opposing components of said critical minimum eigenenergies together with said corresponding counter risks and risks exhibited by said minimum risk quadratic classification system are symmetrically balanced with each other about said geometric center of said dual locus, and wherein said statistical fulcrum is located at said center of said total allowed eigenenergy and said minimum expected risk of said minimum risk quadratic classification system, wherein said minimum risk quadratic classification system satisfies a state of statistical equilibrium, wherein said total allowed eigenenergy and said expected risk of said minimum risk quadratic classification system are minimized, and wherein said minimum risk quadratic classification system exhibits the minimum probability of error for classifying said N feature vectors that belong to said class and said N feature vectors that belong to said other class and said unknown feature vectors related to said data set of said class and said data set of said other class;
• determining M different ensembles of M−1 different discriminant functions of M−1 different minimum risk quadratic classification systems using said M different data sets, said determination of said M different ensembles of M−1 different discriminant functions of M−1 different minimum risk quadratic classification systems being performed by performing said steps of determining M ensembles of M−1 discriminant functions of M−1 minimum risk quadratic classification systems;
• determining a fused discriminant function of a fused M-class minimum risk quadratic classification system using said M ensembles of M−1 discriminant functions of M−1 minimum risk quadratic classification systems and said M different ensembles of M−1 different discriminant functions of M−1 different minimum risk quadratic classification systems, said determination of said fused discriminant function of said fused M-class minimum risk quadratic classification system being performed by using said processors of said computer system to sum said M ensembles of M−1 discriminant functions of M−1 minimum risk quadratic classification systems and said M different ensembles of M−1 different discriminant functions of M−1 different minimum risk quadratic classification systems;
• and
• determining which of said M classes said unknown feature vectors and said unknown different feature vectors belong to using said fused discriminant function of said fused M-class minimum risk quadratic classification system, said determination of said classes of said unknown feature vectors and said unknown different feature vectors being performed by using said processors of said computer system to apply said fused discriminant function of said fused M-class minimum risk quadratic classification system to said unknown feature vectors and said unknown different feature vectors, wherein said fused discriminant function determines likely locations of said unknown feature vectors and said unknown different feature vectors and identifies said decision regions related to said M classes that said unknown feature vectors and said unknown different feature vectors are located within, wherein said fused discriminant function recognizes said classes of said unknown feature vectors and said unknown different feature vectors, and wherein said fused M-class minimum risk quadratic classification system decides which of said M classes said unknown feature vectors and said unknown different feature vectors belong to and thereby classifies said unknown feature vectors and said unknown different feature vectors.

A computer-implemented method of using feature vectors and machine learning algorithms to determine a discriminant function of a minimum risk quadratic classification system that classifies said feature vectors into two classes and using said discriminant function of said minimum risk quadratic classification system to determine a classification error rate and a measure of overlap between distributions of said feature vectors, said method comprising:

• receiving an N×d data set of feature vectors within a computer system, wherein N is a number of feature vectors, d is a number of vector components in each feature vector, and each one of said N feature vectors is labeled with information that identifies which of two classes each one of said N feature vectors belongs to, and wherein each said feature vector is defined by a d-dimensional vector of numerical features, wherein said numerical features are extracted from digital signals;
• receiving an N×d test data set of test feature vectors related to said data set within said computer system, wherein N is a number of test feature vectors, d is a number of vector components in each test feature vector, and each one of said N test feature vectors is labeled with information that identifies which of said two classes each one of said N test feature vectors belongs to;
• determining a kernel matrix using said data set, said determination of said kernel matrix being performed by using processors of said computer system to calculate a matrix of all possible inner products of signed reproducing kernels of said N feature vectors, wherein a reproducing kernel of a feature vector replaces said feature vector with a curve that contains first and second degree vector components, and wherein each one of said reproducing kernels of said N feature vectors has a sign of +1 or −1 that identifies which of said two classes each one of said N feature vectors belongs to, and using said processors of said computer system to calculate a regularized kernel matrix from said kernel matrix;
• determining scale factors of a geometric locus of signed and scaled reproducing kernels of extreme points using said regularized kernel matrix, wherein said extreme points are located within overlapping regions or near tail regions of distributions of said N feature vectors, said determination of said scale factors being performed by using said processors of said computer system to determine a solution of a dual optimization problem, wherein said scale factors and said geometric locus satisfy a system of fundamental locus equations of binary classification, subject to geometric and statistical conditions for a minimum risk quadratic classification system in statistical equilibrium, and wherein said scale factors determine conditional densities for said extreme points and also determine critical minimum eigenenergies exhibited by scaled extreme vectors on said geometric locus, wherein said critical minimum eigenenergies determine conditional probabilities of said extreme points and also determine corresponding counter risks and risks of a minimum risk quadratic classification system, wherein said counter risks are associated with right decisions and said risks are associated with wrong decisions of said minimum risk quadratic classification system, and wherein said geometric locus determines the principal eigenaxis of the decision boundary of said minimum risk quadratic classification system, wherein said principal eigenaxis exhibits symmetrical dimensions and density, wherein said conditional probabilities and said critical minimum eigenenergies exhibited by said minimum risk quadratic classification system are symmetrically concentrated within said principal eigenaxis, and wherein counteracting and opposing components of said critical minimum eigenenergies exhibited by corresponding components of said scaled extreme vectors on said geometric locus together with said corresponding counter risks and risks exhibited by said minimum risk quadratic classification system are symmetrically balanced with each other about the geometric center of said principal eigenaxis, wherein the center of total allowed eigenenergy and minimum expected risk of said minimum risk quadratic classification system is located at the geometric center of said geometric locus, and wherein said geometric locus determines a primal representation of a dual locus of likelihood components and principal eigenaxis components, wherein said likelihood components and said principal eigenaxis components are symmetrically distributed over either side of the axis of said dual locus, wherein a statistical fulcrum is placed directly under the center of said dual locus, and wherein said likelihood components of said dual locus determine conditional likelihoods for said extreme points, and wherein said principal eigenaxis components of said dual locus determine an intrinsic coordinate system of geometric loci of a quadratic decision boundary and corresponding decision borders that jointly partition the decision space of said minimum risk quadratic classification system into symmetrical decision regions;
• determining said extreme vectors on said geometric locus using the vector of said scale factors, said determination of said extreme vectors being performed by using said processors of said computer system to identify said scale factors that exceed zero by a small threshold, and using said processors of said computer system to determine a sign vector of signs associated with said extreme vectors using said data set, and compute the average sign using said sign vector;
• determining a locus of risk for said minimum risk quadratic classification system using said reproducing kernels of said extreme vectors and said signed reproducing kernels of said N feature vectors and said vector of scale factors, said determination of said locus of risk being performed by using said processors of said computer system to calculate a matrix of inner products between said signed reproducing kernels of said N feature vectors and said reproducing kernels of said extreme vectors, and multiply said matrix by said vector of scale factors, and compute the average risk for said minimum risk quadratic classification system using said locus of risk;
• determining a discriminant locus for said minimum risk quadratic classification system using said geometric locus, said determination of said discriminant locus being performed by using said processors of said computer system to calculate a matrix of inner products between said signed reproducing kernels of said N feature vectors and said reproducing kernels of said N feature vectors and said N test feature vectors, and multiply said matrix by said vector of scale factors;
• determining the discriminant function of said minimum risk quadratic classification system, using said average risk and said average sign and said discriminant locus, said determination of said discriminant function of said minimum risk quadratic classification system being performed by using said processors of said computer system to subtract said average risk from sum of said discriminant locus and said average sign, wherein said discriminant function of said minimum risk quadratic classification system satisfies said system of fundamental locus equations of binary classification, and wherein said discriminant function of said minimum risk quadratic classification system determines likely locations of said N feature vectors and said N test feature vectors and also determines said geometric loci of said quadratic decision boundary and said corresponding decision borders that jointly partition said extreme points into said symmetrical decision regions, wherein said symmetrical decision regions span said overlapping regions or said tail regions of said distributions of said N feature vectors, and wherein said discriminant function of said minimum risk quadratic classification system satisfies said quadratic decision boundary in terms of a critical minimum eigenenergy and said minimum expected risk, wherein said counteracting and opposing components of said critical minimum eigenenergies exhibited by said corresponding components of said scaled extreme vectors on said geometric locus associated with said corresponding counter risks and risks exhibited by said minimum risk quadratic classification system are symmetrically distributed over said axis of said dual locus, on equal sides of said statistical fulcrum located at said geometric center of said dual locus, wherein said counteracting and opposing components of said critical minimum eigenenergies together with said corresponding counter risks and risks exhibited by said minimum risk quadratic classification system are symmetrically balanced with each other about said geometric center of said dual locus, and wherein said statistical fulcrum is located at said center of said total allowed eigenenergy and said minimum expected risk of said minimum risk quadratic classification system, wherein said minimum risk quadratic classification system satisfies a state of statistical equilibrium, wherein said total allowed eigenenergy and said expected risk of said minimum risk quadratic classification system are minimized, and wherein said minimum risk quadratic classification system exhibits the minimum probability of error for classifying said N feature vectors and said N test feature vectors related to said data set;
• determining which of said two classes said N feature vectors belong to using said discriminant function of said minimum risk quadratic classification system, said determination of said classes of said N feature vectors being performed by using said processors of said computer system to apply said discriminant function of said minimum risk quadratic classification system to said N feature vectors, wherein said discriminant function determines likely locations of said N feature vectors and identifies said decision regions related to said two classes that said N feature vectors are located within, wherein said discriminant function recognizes said classes of said N feature vectors, and wherein said minimum risk quadratic classification system decides which of said two classes said N feature vectors belong to belong to and thereby classifies said N feature vectors;
• determining an in-sample classification error rate for said two classes of feature vectors, said determination of said error rate being performed by using said processors of said computer system to calculate the average number of wrong decisions made by said minimum risk quadratic classification system for classifying said N features vectors;
• determining which of said two classes said N test feature vectors belong to using said discriminant function of said minimum risk quadratic classification system, said determination of said classes of said N test feature vectors being performed by using said processors of said computer system to apply said discriminant function of said minimum risk quadratic classification system to said N test feature vectors, wherein said discriminant function determines likely locations of said N test feature vectors and identifies said decision regions related to said two classes that said N test feature vectors are located within, wherein said discriminant function recognizes said classes of said N test feature vectors, and wherein said minimum risk quadratic classification system decides which of said two classes said N test feature vectors belong to and thereby classifies said N test feature vectors;
• determining an out-of-sample classification error rate for said two classes of feature vectors, said determination of said error rate being performed by using said processors of said computer system to calculate the average number of wrong decisions made by said minimum risk quadratic classification system for classifying said N test features vectors;
• determining a classification error rate for said two classes of feature vectors, said determination of said classification error rate being performed by using said processors of said computer system to average said in-sample classification error rate and said out-of-sample classification error rate;
• and
• determining a measure of overlap between distributions of feature vectors for said two classes of feature vectors using said N feature vectors and said extreme vectors, said determination of said measure of overlap being performed by using said processors of said computer system to calculate the ratio of the number of said extreme vectors to the number of said N feature vectors, wherein said ratio determines said measure of overlap.

A computer-implemented method of using feature vectors and machine learning algorithms to determine a discriminant function of a minimum risk quadratic classification system that classifies collections of said feature vectors into two classes and using said discriminant function of said minimum risk quadratic classification system to determine if distributions of said collections of said feature vectors are homogenous distributions, said method comprising:

• receiving an N×d data set of feature vectors within a computer system, wherein N is a number of feature vectors, d is a number of vector components in each feature vector, and each one of said N feature vectors is labeled with information that identifies which of two collections each one of said N feature vectors belongs to, and wherein each said feature vector is defined by a d-dimensional vector of numerical features, wherein said numerical features are extracted from digital signals;
• determining a kernel matrix using said data set, said determination of said kernel matrix being performed by using processors of said computer system to calculate a matrix of all possible inner products of signed reproducing kernels of said N feature vectors, wherein a reproducing kernel of a feature vector replaces said feature vector with a curve that contains first and second degree vector components, and wherein each one of said reproducing kernels of said N feature vectors has a sign of +1 or −1 that identifies which of said two collections each one of said N feature vectors belongs to, and using said processors of said computer system to calculate a regularized kernel matrix from said kernel matrix;
• determining scale factors of a geometric locus of signed and scaled reproducing kernels of extreme points using said regularized kernel matrix, wherein said extreme points are located within overlapping regions or near tail regions of distributions of said N feature vectors, said determination of said scale factors being performed by using said processors of said computer system to determine a solution of a dual optimization problem, wherein said scale factors and said geometric locus satisfy a system of fundamental locus equations of binary classification, subject to geometric and statistical conditions for a minimum risk quadratic classification system in statistical equilibrium, and wherein said scale factors determine conditional densities for said extreme points and also determine critical minimum eigenenergies exhibited by scaled extreme vectors on said geometric locus, wherein said critical minimum eigenenergies determine conditional probabilities of said extreme points and also determine corresponding counter risks and risks of a minimum risk quadratic classification system, wherein said counter risks are associated with right decisions and said risks are associated with wrong decisions of said minimum risk quadratic classification system, and wherein said geometric locus determines the principal eigenaxis of the decision boundary of said minimum risk quadratic classification system, wherein said principal eigenaxis exhibits symmetrical dimensions and density, wherein said conditional probabilities and said critical minimum eigenenergies exhibited by said minimum risk quadratic classification system are symmetrically concentrated within said principal eigenaxis, and wherein counteracting and opposing components of said critical minimum eigenenergies exhibited by corresponding components of said scaled extreme vectors on said geometric locus together with corresponding counter risks and risks exhibited by said minimum risk quadratic classification system are symmetrically balanced with each other about the geometric center of said principal eigenaxis, wherein the center of total allowed eigenenergy and minimum expected risk of said minimum risk quadratic classification system is located at the geometric center of said geometric locus, and wherein said geometric locus determines a primal representation of a dual locus of likelihood components and principal eigenaxis components, wherein said likelihood components and said principal eigenaxis components are symmetrically distributed over either side of the axis of said dual locus, wherein a statistical fulcrum is placed directly under the center of said dual locus, and wherein said likelihood components of said dual locus determine conditional likelihoods for said extreme points, and wherein said principal eigenaxis components of said dual locus determine an intrinsic coordinate system of geometric loci of a quadratic decision boundary and corresponding decision borders that jointly partition the decision space of said minimum risk quadratic classification system into symmetrical decision regions;
• determining said extreme vectors on said geometric locus using the vector of said scale factors, said determination of said extreme vectors being performed by using said processors of said computer system to identify said scale factors that exceed zero by a small threshold, and using said processors of said computer system to determine a sign vector of signs associated with said extreme vectors using said data set, and compute the average sign using said sign vector;
• determining a locus of risk for said minimum risk quadratic classification system using said reproducing kernels of said extreme vectors and said signed reproducing kernels of said N feature vectors and said vector of scale factors, said determination of said locus of risk being performed by using said processors of said computer system to calculate a matrix of inner products between said signed reproducing kernels of said N feature vectors and said reproducing kernels of said extreme vectors, and multiply said matrix by said vector of scale factors, and compute the average risk for said minimum risk quadratic classification system using said locus of risk;
• determining a discriminant locus for said minimum risk quadratic classification system using said geometric locus, said determination of said discriminant locus being performed by using said processors of said computer system to calculate a matrix of inner products between said signed reproducing kernels of said N feature vectors and said reproducing kernels of said N feature vectors , and multiply said matrix by said vector of scale factors;
• determining the discriminant function of said minimum risk quadratic classification system, using said average risk and said average sign and said discriminant locus, said determination of said discriminant function of said minimum risk quadratic classification system being performed by using said processors of said computer system to subtract said average risk from sum of said discriminant locus and said average sign, wherein said discriminant function of said minimum risk quadratic classification system satisfies said system of fundamental locus equations of binary classification, and wherein said discriminant function of said minimum risk quadratic classification system determines likely locations of said N feature vectors and also determines said geometric loci of said quadratic decision boundary and said corresponding decision borders that jointly partition said extreme points into said symmetrical decision regions, wherein said symmetrical decision regions span said overlapping regions or said tail regions of said distributions of said N feature vectors, and wherein said discriminant function of said minimum risk quadratic classification system satisfies said quadratic decision boundary in terms of a critical minimum eigenenergy and said minimum expected risk, wherein said counteracting and opposing components of said critical minimum eigenenergies exhibited by said corresponding components of said scaled extreme vectors on said geometric locus associated with said corresponding counter risks and risks exhibited by said minimum risk quadratic classification system are symmetrically distributed over said axis of said dual locus, on equal sides of said statistical fulcrum located at said geometric center of said dual locus, wherein said counteracting and opposing components of said critical minimum eigenenergies together with said corresponding counter risks and risks exhibited by said minimum risk quadratic classification system are symmetrically balanced with each other about said geometric center of said dual locus, and wherein said statistical fulcrum is located at said center of said total allowed eigenenergy and said minimum expected risk of said minimum risk quadratic classification system, wherein said minimum risk quadratic classification system satisfies a state of statistical equilibrium, wherein said total allowed eigenenergy and said expected risk of said minimum risk quadratic classification system are minimized, and wherein said minimum risk quadratic classification system exhibits the minimum probability of error for classifying said N feature vectors that belong to said two collections of said feature vectors;
• determining which of said two collections said N feature vectors belong to using said discriminant function of said minimum risk quadratic classification system, said determination of said collections of said N feature vectors being performed by using said processors of said computer system to apply said discriminant function of said minimum risk quadratic classification system to said N feature vectors, wherein said discriminant function determines likely locations of said N feature vectors and identifies said decision regions related to said two collections that said N feature vectors are located within, wherein said discriminant function recognizes said collections of said N feature vectors, and wherein said minimum risk quadratic classification system decides which of said two collections said N feature vectors belong to belong to and thereby classifies said N feature vectors;
• determining an in-sample classification error rate for said two collections of feature vectors, said determination of said error rate being performed by using said processors of said computer system to calculate the average number of wrong decisions made by said minimum risk quadratic classification system for classifying said N features vectors;
• determining a measure of overlap between said distributions of said N feature vectors for said two collections of feature vectors using said N feature vectors and said extreme vectors, said determination of said measure of overlap being performed by using said processors of said computer system to calculate the ratio of the number of said extreme vectors to the number of said N feature vectors, wherein said ratio determines said measure of overlap;
• and
• determining if said distributions of said two collections of said N feature vectors are homogenous distributions using said in-sample classification error rate and said measure of overlap, wherein said distributions of said N feature vectors are homogenous distributions if said measure of overlap has an approximate value of one and said in-sample classification error rate has an approximate value of one half.