Back to results
US 2020/0265336 A1Published application
Systems and methods for decomposition of differentiable and non-differentiable models
AssigneeZestFinance, Inc.
InventorsJohn Wickens Lamb Merrill, Geoffrey Michael Ward, Sean Javad Kamkar, John Joseph Beahan, JR., Marc Frederick Eberstein, Jose Efrain Valentin, Jerome Louis Budzik
Life of this patent
Estimating the term…
Abstract
Systems and methods for understanding diverse machine learning models.
Claims20
A method comprising:
- with a decomposition module, accessing model access information for a model included in a modeling system, the model access information identifying at least one tree structure used by the model;
- with the decomposition module, identifying a reference input data set, an evaluation input data set, and a plurality of features included in at least one of the reference input data set and the evaluation input data set;
- with the decomposition module, determining a feature contribution value for each identified feature, wherein each feature contribution value identifies a contribution by the feature to a difference in output generated by the model for the evaluation input data set, relative to the reference input data set;
- generating model explanation information by using the determined feature contribution values, wherein the model explanation information is generated by using at least one of a model evaluation module, an output explanation module, and a model monitoring module, and wherein the model explanation information explains the difference in output generated by the model for the evaluation input data set, relative to the reference input data set, and wherein the model explanation information includes information relating to a model-based decision;
- with a user interface system, providing the generated model explanation information to an operator device,
- wherein, determining a feature contribution value for each identified feature comprises:
- identifying a path between the reference input data set and the evaluation input data set,
- identifying model decision boundary points along the path by using the model access information,
- identifying a plurality of path segments by segmenting the path at each identified boundary point,
- for each segment, determining a segment contribution value for each feature by determining an integral of a gradient for the model along the segment,
- for each boundary point, determining a boundary point contribution value for the boundary point, and assigning the boundary contribution value to at least one of the features, and
- for each endpoint of the path between the reference input data set and the evaluation input data set, assigning a contribution of each feature at the endpoint
- for each feature, combining the feature's segment contribution values and any boundary point and endpoint contribution values assigned to the feature to generate the feature contribution value for the feature with respect to at least two data points
10 more elements
A system comprising:
- a modeling system that includes a model;
- a decomposition module communicatively coupled to the modeling system;
- a model evaluation module communicatively coupled to the decomposition module;
- an output explanation module communicatively coupled to the decomposition module;
- a model monitoring module communicatively coupled to the decomposition module;
- and
- a user interface system communicatively coupled to at least one of the decomposition module, the evaluation module, the explanation module, and the monitoring module,
- wherein the decomposition module is constructed to:
- access model access information for the model included in the modeling system, the model access information identifying at least one tree structure used by the model,
- identify a reference input data set, an evaluation input data set, and a plurality of features included in at least one of the reference input data set and the evaluation input data set,
- determine a feature contribution value for each identified feature, wherein each feature contribution value identifies a contribution by the feature to output generated by the model for the evaluation input data set, relative to the reference input data set,
- wherein at least one of the model evaluation module, the output explanation module, and the model monitoring module is constructed to generate model explanation information by using the determined feature contribution values,
- wherein the model explanation information explains the model for the evaluation input data set relative to the reference input data set,
- wherein the user interface system is constructed to provide the generated model explanation information to an operator device, and
- wherein, determining a feature contribution value for each identified feature comprises:
- identifying a path between the reference input data set and the evaluation input data set,
- identifying model decision boundary points along the path by using the model access information,
- identifying a plurality of path segments by segmenting the path at each identified boundary point,
- for each segment, determining segment contribution values for each feature by determining an integral of a gradient for the model along the segment,
- for each boundary point, determining a boundary point contribution value for the boundary point, and assigning the boundary contribution value to one of the features, and
- for each feature, combining the feature's segment contribution values and any boundary point contribution values assigned to the feature to generate the feature contribution value for the feature.
18 more elements
Dependencies are inferred from the claim text.