Classification based on prediction of accuracy of multiple data models

A computing device, comprising:

• a processor;
• a plurality of models, at least a subset of the plurality of models configured to generate model outputs responsive to receiving instance data associated with an action or an event, each generated model output representing classification of the action or the event made by each model;
• a plurality of oracles configured to generating confidence values of corresponding models based at least on the generated model outputs, each of the oracles trained to predict accuracy of a corresponding model for the instance data;
• an output selector configured to select one of the plurality of models likely to provide an accurate model output based on the model outputs and the confidence values for the instance data.

A non-transitory computer-readable storage medium configured to store instructions, when executed by a processor, cause the processor to:

• generate model outputs by at least a subset of a plurality of models responsive to receiving instance data associate with an action or an event, each generated model output representing classification of the action or the event made by a corresponding model based on the instance data;
• generate confidence values of the models for the instance data at oracles based at least on the generated model outputs, each of the oracles trained to predict accuracy of a corresponding model for the instance data;
• and
• select a model likely to provide a more accurate model output based on the model outputs and the confidence values for the instance data.