Systems and techniques for determining the predictive value of a feature

A predictive modeling apparatus comprising:

• a memory configured to store processor-executable instructions;
• and
• a processor configured to execute the processor-executable instructions, wherein executing the processor-executable instructions causes the apparatus to perform steps including:
• (a) performing a plurality of predictive modeling procedures, wherein each of the predictive modeling procedures is associated with a predictive model, and wherein performing each modeling procedure comprises fitting the associated predictive model to an initial dataset representing an initial prediction problem;
• (b) determining a first respective accuracy score of each of the fitted predictive models, wherein the first accuracy score of each fitted model represents an accuracy with which the fitted model predicts one or more outcomes of the initial prediction problem;
• (c) shuffling values of a feature across respective observations included in the initial dataset, thereby generating a modified dataset representing a modified prediction problem;
• (d) determining a second respective accuracy score of each of the fitted predictive models, wherein the second accuracy score of each fitted model represents an accuracy with which the fitted model predicts one or more outcomes of the modified prediction problem;
• and
• (e) determining a respective model-specific predictive value of the feature for each of the fitted models, wherein the model-specific predictive value of the feature for each fitted model is based on the first and second accuracy scores of the fitted model.

An article of manufacture having computer-readable instructions stored thereon that, when executed by a processor, cause the processor to perform operations including:

• (a) performing a plurality of predictive modeling procedures, wherein each of the predictive modeling procedures is associated with a predictive model, and wherein performing each modeling procedure comprises fitting the associated predictive model to an initial dataset representing an initial prediction problem;
• (b) determining a first respective accuracy score of each of the fitted predictive models, wherein the first accuracy score of each fitted model represents an accuracy with which the fitted model predicts one or more outcomes of the initial prediction problem;
• (c) shuffling values of a feature across respective observations included in the initial dataset, thereby generating a modified dataset representing a modified prediction problem;
• (d) determining a second respective accuracy score of each of the fitted predictive models, wherein the second accuracy score of each fitted model represents an accuracy with which the fitted model predicts one or more outcomes of the modified prediction problem;
• and
• (e) determining a respective model-specific predictive value of the feature for each of the fitted models, wherein the model-specific predictive value of the feature for each fitted model is based on the first and second accuracy scores of the fitted model.