Robustness score for an opaque model

A system comprising:

• a processor;
• a data bus coupled to the processor;
• and
• a non-transitory, computer-readable storage medium embodying computer program code, the non-transitory, computer-readable storage medium being coupled to the data bus, the computer program code interacting with a plurality of computer operations and comprising instructions executable by the processor and configured for:
• receiving a subject data point;
• classifying the data point via a model trainer, the classifying providing a classified data point;
• identifying a counterfactual using the classified data point, the counterfactual comprising another datapoint, the identifying being performed by an augmented intelligence system assurance engine of an augmented intelligence assurance framework, the another data point being close to the subject data point, the another data point resulting in production of a different outcome when provided to a model when compared to an outcome resulting from the subject data point being provided to the model;
• and,
• providing the counterfactual to a destination.

A non-transitory, computer-readable storage medium embodying computer program code, the computer program code comprising computer executable instructions configured for:

• receiving a subject data point;
• classifying the data point via a model trainer, the classifying providing a classified data point;
• identifying a counterfactual using the classified data point, the counterfactual comprising another datapoint, the identifying being performed by an augmented intelligence system assurance engine of an augmented intelligence assurance framework, the another data point being close to the subject data point, the another data point resulting in production of a different outcome when provided to a model when compared to an outcome resulting from the subject data point being provided to the model;
• and,
• providing the counterfactual to a destination.