US 12,585,998 B2Granted

Determining quality of machine learning model output

AssigneeCapital One Services, LLC

InventorsSamuel Sharpe, Brian Barr, Isha Hameed, Justin Au-Yeung, Areal Tal, Daniel Barcklow

Life of this patent

Estimating the term…

Loading patent…

Abstract

In some aspects, a computing system may generate uninformative features that may be added to a dataset of real features to use as a baseline for determining the quality of an explanation of model output. The uninformative features may be features that do not correlate with what a model is tasked with predicting (e.g., the uninformative features may be random values), and the real features may be informative and correlate with what the model is tasked with predicting (e.g., variables of a dataset sample). A machine learning model may be trained on a dataset that includes both the real features and the uninformative features. The computing system may generate feature attributions for model output, which may include feature attributions for the uninformative features and the real features in the dataset.

Claims20

Dependent claims

A system for facilitating efficacy of explanations of machine learning model output through use of uninformative features, the system comprising:

one or more processors and one or more non-transitory media having instructions recorded thereon that, when executed by the one or more processors, cause operations comprising:
obtaining a dataset comprising a set of real features and a set of samples, wherein each sample comprises a value for each feature in the set of real features;
generating, based on the dataset, a set of uninformative features comprising one or more values for each sample in the set of samples, wherein the set of uninformative features are generated such that the uninformative features do not indicate correct classes of samples in the set of samples, wherein the set of uninformative features are combined with the set of real features to form a combined set of features;
in connection with monitoring of a deployed model and detecting degraded performance of the deployed model, training, based on the dataset and the combined set of features, a machine learning model;
generating a first local explanation associated with a first sample of the dataset and first output of the machine learning model, wherein the first local explanation indicates a first ranking of the combined set of features;
determining, based on the first local explanation, that a first uninformative feature of the set of uninformative features is ranked higher than a first real feature of the set of real features, wherein the first uninformative feature ranking higher than the first real feature indicates that the first uninformative feature is more influential to the machine learning model;
and
based on the first uninformative feature ranking higher than the first real feature, removing the first real feature from the first local explanation to increase a trustworthiness of the first local explanation.

5 more elements

A method for facilitating efficacy of explanations of machine learning model output through use of uninformative features, the method comprising:

obtaining a dataset comprising a set of real features and a set of samples, wherein each sample comprises a value for each feature in the set of real features;
generating a set of uninformative features comprising one or more values for each sample in the set of samples, wherein the set of uninformative features are combined with the set of real features to form a combined set of features;
in connection with monitoring of a deployed model and detecting degraded performance of the deployed model, training, based on the dataset and the combined set of features, a machine learning model;
generating a first local explanation associated with a first sample of the dataset and first output of the machine learning model, wherein the first local explanation indicates a first ranking of the combined set of features;
determining, based on the first local explanation, that a first uninformative feature of the set of uninformative features is ranked higher than a first real feature of the set of real features, wherein the first uninformative feature ranking higher than the first real feature indicates that the first uninformative feature is more influential to the machine learning model;
and
based on the first uninformative feature ranking higher than the first real feature, removing the first real feature from the first local explanation.

4 more elements

One or more non-transitory computer-readable media comprising instructions that, when executed by one or more processors, cause operations comprising:

obtaining a dataset comprising a set of real features and a set of samples, wherein each sample comprises a value for each feature in the set of real features;
generating a set of uninformative features comprising one or more values for each sample in the set of samples, wherein the set of uninformative features are combined with the set of real features to form a combined set of features;
in connection with monitoring of a deployed model and detecting degraded performance of the deployed model, training, based on the dataset and the combined set of features, a machine learning model;
generating a first local explanation associated with a first sample of the dataset and first output of the machine learning model, wherein the first local explanation indicates a first ranking of the combined set of features;
determining, based on the first local explanation, that a first uninformative feature of the set of uninformative features is ranked higher than a first real feature of the set of real features, wherein the first uninformative feature ranking higher than the first real feature indicates that the first uninformative feature is more influential to the machine learning model;
and
based on the first uninformative feature ranking higher than the first real feature, removing the first real feature from the first local explanation.

4 more elements

Dependencies are inferred from the claim text.