Maximizing mutual information between observations and hidden states to minimize classification errors
AssigneeMicrosoft Corporation
InventorsNuria M. Oliver, Ashutosh Garg
Life of this patent
Estimating the term…
Abstract
The present invention relates to a system and methodology to facilitate machine learning and predictive capabilities in a processing environment. In one aspect of the present invention, a Mutual Information Model is provided to facilitate predictive state determinations in accordance with signal or data analysis, and to mitigate classification error. The model parameters are computed by maximizing a convex combination of the mutual information between hidden states and the observations and the joint likelihood of states and observations in training data. Once the model parameters have been learned, new data can be accurately classified.
Claims36
A learning system, comprising:
- a prediction component to determine one or more states based in part upon previous training data and sampled data;
- and
- a classification model that cooperates with the prediction component to determine the one or more states, the classification model having at least one of observed data and at least one hidden state, the classification model at least one of maximizes the likelihood of the observed data and the mutual information between the at least one hidden state and the observed data in order to mitigate classification error associated with the model.
A method to mitigate classification errors, comprising:
- determining a conditional entropy relationship versus an optimal classification error for a model;
- estimating the model from data;
- and
- optimizing the model parameters by trading-off a maximum likelihood criterion and a maximum mutual information criterion to mitigate classification errors associated with the model.
A system to facilitate automated learning, comprising:
- means for automatically determining one or more hidden states;
- means for modeling observed data and at least one hidden state;
- means for optimizing a convex combination of a likelihood of the observed data and a mutual information between the at least one state and the observed data in order to mitigate classification error.
A signal to communicate prediction data between at least two nodes, comprising:
- a first data packet comprising:
- a prediction component to transmit one or more states between at least two nodes;
- and
- a classification component to determine the one or more states, the classification component optimizing a parameter that balances a maximum likelihood criterion and a maximum mutual information criterion to mitigate classification errors associated with the classification component.
A computer-readable medium having stored thereon a data structure, comprising:
- a first data field containing training data associated with a learning algorithm;
- and
- a second data field containing the parameters of a model that balances a maximum likelihood criterion and a maximum mutual information criterion to mitigate classification errors associated with a classifier.
Dependencies are inferred from the claim text.