System and method for combining multiple learning agents to produce a prediction method

A computer-implemented method of producing a prediction method, the method comprising:

• providing training data related to a problem for which a prediction method is sought, the training data represented by features with values stored in a computer-readable medium;
• providing at least two learning agents having a same parameter list but different parameter values;
• training the learning agents on the training data, each agent producing in response to the data a prediction method based on the agent's parameter values;
• determining a fitness measure for each learning agent based on the prediction method the agent produces;
• selecting the parameter values of at least one of the learning agents based on the agent's fitness measure;
• introducing variation into the selected parameter values;
• defining another learning agent using the varied parameter values;
• and
• again training the learning agents on the training data to cause a learning agent to produce a prediction method based on the varied parameter values, wherein the prediction method comprises a set of computer-executable instructions stored in a computer-readable medium and returns a result.

A computer-implemented method of producing a prediction method, the method comprising the following steps:

• providing training data related to a problem for which a prediction method is sought, the training data represented by features with values stored in a computer-readable medium;
• providing a plurality of learning agents of which at least two have a same parameter list but different parameter values, the learning agents including input representations;
• training the learning agents on the training data, each agent producing in response to the data a prediction method based on the agent's parameter values and input representation;
• determining a fitness measure for each learning agent based on the prediction method the agent produces;
• selecting the parameter values of at least one of the learning agents based on the agent's fitness measure;
• introducing variation into the selected parameter values;
• defining another learning agent using the varied parameter values;
• extracting feature combinations from the prediction methods produced by the learning agents, wherein the extracting comprises combining at least two of the features;
• modifying the input representation of a learning agent by including in the input representation a feature combination extracted from another learning agent;
• and
• again training the learning agents on the training data to cause a learning agent to produce a prediction method based on the varied parameter set and the modified input representation, wherein the prediction method comprises a set of computer-executable instructions stored in a computer-readable medium and returns a result.

A computer-implemented system for producing a prediction method for a problem, comprising:

• a training data set related to a problem for which a prediction method is sought, the training data set represented by features with values stored in a computer-readable medium;
• at least two learning agents, the agents including input representations;
• and
• at least one computer programmed for:
• training the learning agents on the data set, each agent producing in response to the data a prediction method based on the agent's input representation;
• extracting feature combinations from the prediction methods produced by the learning agents, wherein the extracting comprises combining at least two of the features;
• modifying the input representation of a learning agent by including in the input representation a feature combination extracted from another learning agent;
• and
• again training the learning agents on the data set to cause a learning agent to produce a prediction method based on the agent's modified input representation, wherein the prediction method comprises a set of computer-executable instructions stored in a computer-readable medium and returns a result.

A general purpose computer for producing a prediction method for a problem, the system comprising:

• computer processor means for processing data;
• storage means for storing data in a storage medium, wherein the stored data comprises training data related to a problem for which a prediction method is sought, the training data represented by features with values, and the stored data further comprises at least two learning agents, the learning agents including input representations;
• first means for training the learning agents on the data set, at least two of the agents producing in response to the data a prediction method based on the agent's input representation;
• second means for extracting feature combinations from the prediction methods produced by the learning agents;
• third means for modifying the input representation of a learning agent by including in the input representation a feature combination extracted from another learning agent;
• and
• fourth means for again training the learning agents on the data set to cause a learning agent to produce a computer-executable prediction method based on the agent's modified input representation.

A computer-readable medium comprising computer-executable instructions for performing at least the following:

• for training data comprising a set of features and indicating a set of observations comprising values for the features and values indicating observations of results related to the feature values, training a plurality of software-based learning agents to generate a plurality of computer-executable functions, wherein the computer-executable functions provide predicted values given a set of values for at least a subset of the features;
• selecting a subset of the features incorporated into a first of the computer-executable functions generated by a first software-based learning agent and combining the features into a feature combination;
• based on the feature combination and the training data, generating modified training data comprising the feature combination and values for the feature combination;
• and
• training the software-based learning agents on the modified training data to generate a plurality of second-generation computer-executable functions, wherein the second-generation computer-executable functions provide predicted values given a set of values for at least the feature combination;
• wherein the features of the feature combination are extracted from the first computer-executable function originating from a first software-based learning agent and the feature combination is incorporated into training data from which a second software-based learning agent generates a second, second-generation, computer-executable function.