Determining and selecting prediction models over multiple points in time

One or more computer-readable storage media collectively containing computer program code that, when executed by operation of one or more computer processors, performs an operation comprising:

• receiving a plurality of models trained to generate predictions at each of a plurality of intervals;
• determining a respective prediction accuracy of each respective model of the plurality of models for a first interval of the plurality of intervals by processing labeled evaluation data using the respective model;
• and
• generating a model ensemble specifying one or more of the plurality of models for each of the plurality of intervals, comprising:
• selecting, for the first interval, a first model of the plurality of models based on (i) the respective prediction accuracies and (ii) at least one non-error metric.

A system comprising:

• one or more computer processors;
• and
• one or more memories collectively containing one or more programs which when executed by the one or more computer processors performs an operation, the operation comprising:
• receiving a plurality of models trained to generate predictions at each of a plurality of intervals;
• determining a respective prediction accuracy of each respective model of the plurality of models for a first interval of the plurality of intervals by processing labeled evaluation data using the respective model;
• and
• generating a model ensemble specifying one or more of the plurality of models for each of the plurality of intervals, comprising:
• selecting, for the first interval, a first model of the plurality of models based on (i) the respective prediction accuracies and (ii) at least one non-error metric.