US 2019/0156216 A1Published application

Machine learning model interpretation

AssigneeAdobe Inc.

InventorsPiyush Gupta, Nikaash Puri, Balaji Krishnamurthy

Life of this patent

Estimating the term…

Loading patent…

Abstract

A technique is disclosed for generating class level rules that globally explain the behavior of a machine learning model, such as a model that has been used to solve a classification problem. Each class level rule represents a logical conditional statement that, when the statement holds true for one or more instances of a particular class, predicts that the respective instances are members of the particular class. Collectively, these rules represent the pattern followed by the machine learning model. The techniques are model agnostic, and explain model behavior in a relatively easy to understand manner by outputting a set of logical rules that can be readily parsed. Although the techniques can be applied to any number of applications, in some embodiments, the techniques are suitable for interpreting models that perform the task of classification. Other machine learning model applications can equally benefit.

Claims20

Dependent claims

A computer-implemented method of interpreting a machine learning model, the method comprising:

receiving, by a processor-based system, data representing a machine learning model, a set of training data, and a set of output classes for classifying a plurality of instances of the training data, each instance representing at least one feature of the training data;
generating, by the processor-based system, a set of instance level conditions by applying each instance and at least one perturbation of the respective instance to the machine learning model and calculating, based on an output of the machine learning model, a contribution of each feature of the respective instance to at least one of the output classes, wherein each instance level condition represents a range of values for each feature having a greatest contribution to the output class;
generating, by the processor-based system, a set of class level rules by applying the instance level conditions for each of the corresponding instances to a genetic algorithm, each class level rule representing a logical conditional statement that, when the statement holds true for one or more instances of a particular class, predicts that the respective instances are members of the particular class;
and
causing, by the processor-based system, a display at least a portion of the set of class level rules to a user.

2 more elements

A computer program product including one or more non-transitory computer readable mediums having instructions encoded thereon that when executed by one or more computer processors cause the one or more computer processors to perform a process for interpreting a machine learning model, the process including

receiving data representing a machine learning model, a set of training data, and a set of output classes for classifying a plurality of instances of the training data, each instance representing at least one feature of the training data;
generating a set of instance level conditions by applying each instance and at least one perturbation of the respective instance to the machine learning model and calculating, based on an output of the machine learning model, a contribution of each feature of the respective instance to at least one of the output classes, wherein each instance level condition represents a range of values for each feature having a greatest contribution to the output class;
generating a set of class level rules by applying the instance level conditions for each of the corresponding instances to a genetic algorithm, each class level rule representing a logical conditional statement that, when the statement holds true for one or more instances of a particular class, predicts that the respective instances are members of the particular class;
and
causing a display of at least a portion of the set of class level rules to a user.

2 more elements

A system for interpreting a machine learning model, the system comprising:

one or more storages, and
one or more processors operatively coupled to the one or more storages, the one or more processors configured to execute instructions stored in the one or more storages that when executed cause the one or more processors to carry out a process including
receive data representing a machine learning model, a set of training data, and a set of output classes tar classifying a plurality of instances of the training data, each instance representing at least one feature of the training data;
generate a set of instance level conditions by applying each instance and at least one perturbation of the respective instance to the machine learning model and calculating, based on an output of the machine learning model, a contribution of each feature of the respective instance to at least one of the output classes, wherein each instance level condition represents a range of values for each feature having a greatest contribution to the output class;
generate a set of class level rules by applying the instance level conditions for each of the corresponding instances to a genetic algorithm, each class level rule representing a logical conditional statement that, when the statement holds true for one or more instances of a particular class, predicts that the respective instances are members of the particular class;
and
cause a display of at least a portion of the set of class level rules to a user.

4 more elements

Dependencies are inferred from the claim text.