Cost-aware non-stationary online learning

A method operating in conjunction with a set of ensemble actions that includes an ensemble action that does not update the ensemble of classifiers or regressors, the method including:

• performing ensemble learning of an ensemble of classifiers or regressors to predict a non-stationary time-varying parameter over a time series of episodes, each episode comprising a time period;
• wherein the ensemble learning includes for an episode of the time series of episodes:
• selecting an ensemble action from a set of ensemble actions based on ensemble action quality values (Q values) for the ensemble actions at an error state for the episode that is indicative of error of the ensemble of classifiers or regressors in predicting the non-stationary time-varying parameter,
• executing the selected ensemble action to update the ensemble of classifiers or regressors,
• computing or retrieving a cost of executing the selected ensemble action,
• computing a reward indicative of how well the updated ensemble of classifiers or regressors predicts the non-stationary time-varying parameter over the episode, and
• updating the Q value for the selected ensemble action at the error state for the episode based on both the reward and the cost of executing the selected ensemble action;
• and the performing of ensemble learning further includes, for an episode of the time series of episodes:
• selecting the ensemble action that does not update the ensemble of classifiers or regressors based the Q values for the ensemble actions at the error state for the episode,
• computing or retrieving a cost of executing the selected ensemble action ensemble action that does not update the ensemble of classifiers or regressors,
• computing a reward indicative of how well the updated ensemble of classifiers or regressors predicts the non-stationary time-varying parameter over the episode, and
• updating the Q value for the selected ensemble action that does not update the ensemble of classifiers or regressors at the error state for the episode based on both the reward and the cost of executing the selected ensemble action that does not update the ensemble of classifiers or regressors;
• wherein performing of ensemble learning is performed by an electronic data processing device programmed to perform the ensemble learning.

A non-transitory storage medium storing instructions readable and executable by a computer to perform processing for an episode comprising a time interval of online ensemble learning of an ensemble of classifiers or regressors to predict a non-stationary time-varying parameter, the processing performed for the episode including:

• selecting an ensemble action from a set of ensemble actions based on ensemble action quality values (Q values) for the ensemble actions at an error state for the episode;
• executing the selected ensemble action;
• after executing the selected ensemble action, computing a reward indicative of how well the ensemble of classifiers or regressors predicts the non-stationary time-varying parameter over the episode;
• computing or retrieving a cost of executing the selected ensemble action;
• and
• updating the Q value for the selected ensemble action at the error state for the episode based on both the computed reward and the cost of executing the selected ensemble action.

An apparatus comprising:

• an electronic data processing device configured to perform online ensemble learning of an ensemble of classifiers or regressors to predict a non-stationary time-varying parameter, the performed online ensemble learning including, on a periodic basis:
• selecting an ensemble action from a set of ensemble actions and executing the selected ensemble action;
• and
• computing a reward indicative of how well the updated ensemble of classifiers or regressors predicts the non-stationary time-varying parameter;
• wherein the selecting is based on both costs of executing the ensemble actions of the set of ensemble actions and on reward values previously computed for ensemble actions of the set of ensemble actions
• wherein, for at least one ensemble action of the set of ensemble actions, the performed online ensemble learning assigns different costs for executing the ensemble action at different error states of the ensemble of classifiers or regressors, wherein the different error states correspond to different levels of prediction error of the ensemble of classifiers or regressors respective to measured values of the non-stationary time-varying parameter.