Back to results
US 2019/0378210 A1Published application
Systems and methods for decomposition of non-differentiable and differentiable models
AssigneeZestFinance, Inc.
InventorsDouglas C. Merrill, Michael Edward Ruberry, Sean Javad Kamkar, Jerome Louis Budzik, John Wickens Lamb Merrill
Life of this patent
Estimating the term…
Abstract
Systems and methods for explaining non-differentiable models and differentiable models.
Claims20
A method comprising:
- with a model explanation system:
- accessing non-differentiable model information of a non-differentiable model;
- selecting a reference population of one or more reference data points;
- selecting a test population of one or more test data points;
- for each test data point, generating a non-differentiable decomposition value for each feature of the test data point, wherein generating a non-differentiable decomposition value comprises:
- generating a SHAP (SHapley Additive exPlanation) value for the corresponding feature for the corresponding test data point by using the non-differentiable model information;
- generating a SHAP value for the corresponding feature for each reference data point by using the non-differentiable model information;
- for each reference data point, generating a difference value between the SHAP value for the test data point and the SHAP value of the reference data point;
- generating the non-differentiable decomposition value by averaging the generated difference values;
- generating explanation information based on at least the generated non-differentiable decomposition values;
- providing the generated explanation information to an external system.
8 more elements
A method comprising:
- with a model explanation system
- accessing non-differentiable model information of a non-differentiable model of an ensemble and accessing differentiable model information of a differentiable model of the ensemble;
- selecting a reference population of one or more reference data points;
- selecting a test population of one or more test data points;
- for each test data point, generating a non-differentiable decomposition value for each feature of the test data point, wherein generating a non-differentiable decomposition value comprises:
- generating a SHAP (SHapley Additive exPlanation) value for the corresponding feature for the corresponding test data point by using the non-differentiable model information;
- generating a SHAP value for the corresponding feature for each reference data point by using the non-differentiable model information;
- for each reference data point, generating a difference value between the SHAP value for the test data point and the SHAP value of the reference data point;
- generating the non-differentiable decomposition value by averaging the generated difference values;
- for each test data point, generating a differentiable decomposition value for each feature of the test data point, wherein generating a differentiable decomposition value comprises:
- for each reference data point, performing an integrated gradients process using the test data point and the reference data point to generate an integrated gradient value by using the differentiable model information of the differentiable model;
- and
- generating the differentiable decomposition value by averaging the generated integrated gradients values
- for each test data point, generating ensemble decomposition values, comprising:
- combining non-differentiable decomposition values with corresponding differentiable decomposition values by using an ensembling function of the ensemble;
- generating explanation information based on at least the generated ensemble decomposition values;
- and
- providing the generated explanation information to an external system.
15 more elements
Dependencies are inferred from the claim text.