Mixtures of Bayesian networks

In a decision support system that receives a set of observed data as an input on a signal-bearing medium, an apparatus for finding the likeliest number of states of hidden discrete variables in a mixture of Bayesian networks comprising plural hypothesis-specific Bayesian networks (HSBNs) having nodes corresponding to hidden and observed variables, each of said nodes storing a structure and a set of parameters representing causal relationships among said nodes, said apparatus comprising:

• a processor;
• a memory having executable instructions stored therein;
• and
• wherein said processor, in response to said executable instructions:
• chooses successive numbers of states of said discrete hidden and observed variables, and for each one of said successive numbers of states:
• initializes said HSBNs;
• conducts, for each one of said HSBNs, a parameter search for a set of changes in said parameters which improves the goodness of said one HSBN in predicting said observed data, and modifies the parameters of said one HSBN accordingly;
• computes, for each one of said HSBNs, a structure score of said one HSBN reflecting the goodness of said one HSBN in predicting said observed data, conducts a structure search for a change in said structure which improves said structure search score, and modifies the structure of said one HSBN accordingly;
• computes a combined score of the mixture of Bayesian networks corresponding to the current number of states of said discrete variables;
• and
• chooses the mixture of Bayesian networks having the best score.

In a decision support system that receives a set of observed data as an input on a signal-bearing medium, an apparatus for converting said set of observed data into complete statistics for training a mixture of Bayesian networks comprising plural hypothesis-specific Bayesian networks (HSBNs) having nodes corresponding to hidden and observed variables, each of said nodes storing a structure and a set of parameters representing causal relationships among said nodes, said apparatus comprising:

• a processor;
• a memory having executable instructions stored therein;
• and
• wherein said processor, in response to instructions stored in memory:
• chooses a number of states of said discrete hidden and observed variables, and initializing said HSBNs;
• conducts a parameter search for each one of said HSBNs for a set of changes in said parameters which improves the goodness of said one HSBN in predicting said observed data, and modifying the parameters of said one HSBN accordingly;
• computes a structure score for each one of said HSBNs reflecting the goodness of said one HSBN in predicting said observed data;
• conducts a structure search for a change in said structure which improves said structure search score, and modifies the structure of said one HSBN accordingly;
• wherein the processor computes a structure score of said one HSBN in that the processor
• computes from said observed data expected complete model sufficient statistics (ECMSS),
• computes from said ECMSS sufficient statistics for said one HSBN,
• computes said structure score from said sufficient statistics;
• wherein said processor computes said ECMSS, in that the processor computes the probability of each combination of the states of the discrete hidden and observed variables;
• forms a vector for each observed case in said set of observed data, each entry in said vector corresponding to a particular one of the combinations of the states of said discrete variables;
• and
• sums the vectors over plural cases of said observed data whereby to render a complete set of information.

In a decision support system that receives a set of observed data as an input on a signal-bearing medium, a method for applying the set of observed data for improving the structure and parameters of a mixture of Bayesian networks comprising plural hypothesis-specific Bayesian networks (HSBNs) having nodes corresponding to hidden and observed variables, each of said nodes storing a set of parameters and structure representing dependence relationships among said nodes, said method comprising:

• storing the mixture of Bayesian networks in a memory;
• receiving the set of observed data as an input on the signal-bearing medium;
• choosing a number of said HSBNs;
• choosing a number of states of said discrete variables, and initializing said HSBNs;
• for each one of said HSBNs conducting a parameter search for a set of changes in said parameters which improves the goodness of said one HSBN in predicting said observed data, and modifying the parameters of said one HSBN accordingly;
• for each one of said HSBNs, computing a structure score of said one HSBN reflecting the goodness of said one HSBN in predicting said observed data, conducting a structure search for a change in said structure which improves said structure search score, and modifying the structure of said one HSBN accordingly.

In a decision support system that receives a set of observed data as an input on a signal-bearing medium, a method for finding the likeliest number of states of hidden discrete variables in a mixture of Bayesian networks comprising plural hypothesis-specific Bayesian networks (HSBNs) having nodes corresponding to hidden and observed variables, each of said nodes storing a structure and a set of parameters representing causal relationships among said nodes, said method comprising:

• storing the mixture of Bayesian networks in a memory;
• receiving the set of observed data as an input on the signal-bearing medium;
• choosing successive numbers of states of said discrete hidden and observed variables, and for each one of said successive numbers of states:
• initializing said HSBNs;
• conducting, for each one of said HSBNs, a parameter search for a set of changes in said parameters which improves the goodness of said one HSBN in predicting said observed data, and modifying the parameters of said one HSBN accordingly;
• computing, for each one of said HSBNs, a structure score of said one HSBN reflecting the goodness of said one HSBN in predicting said observed data, conducting a structure for a change in said structure which improves said structure search score, and modifying the structure of said one HSBN accordingly;
• computing a combined score of the mixture of Bayesian networks corresponding to the current number of states of said discrete variables;
• and
• choosing the mixture of Bayesian networks having the best score.