User explanation guided machine learning

A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computing device to cause the computing device to:

• identify a subset of training examples, from a set of training examples, based on at least one of an uncertainty metric computed for each one of the training examples and an influence metric computed for each one of the training examples, wherein the number of training examples in the subset of training examples is limited to a specified number that is less than the total number of training examples in the set;
• output said subset of training examples to a user;
• obtain, from the user, a user explanation for each training example in said subset of training examples, wherein each of the user explanations identifies at least one part of the corresponding training example;
• and
• train a machine learning model based at least in part on the user explanations, wherein said training comprises prioritizing the identified parts of the training examples in the subset.

A system comprising:

• a memory;
• and
• at least one processor operably coupled to the memory and configured for:
• identifying a subset of training examples, from a set of training examples, based on at least one of an uncertainty metric computed for each one of the training examples and an influence metric computed for each one of the training examples, wherein the number of training examples in the subset of training examples is limited to a specified number that is less than the total number of training examples in the set;
• outputting said subset of training examples to a user;
• obtaining, from the user, a user explanation for each training example in said subset of training examples, wherein each of the user explanations identifies at least one part of the corresponding training example;
• and
• training a machine learning model based at least in part on the user explanations, wherein said training comprises prioritizing the identified parts of the training examples in the sub set.

A computer-implemented method, the method comprising:

• training a machine learning model on a dataset for one or more iterations, wherein the dataset comprises one or more training examples;
• obtaining, from a user, one or more user explanations for a subset of said training examples, wherein the user explanations identify one or more areas of importance for each of the subset of training examples in the subset;
• and
• re-training the machine learning model for one or more additional iterations on said dataset, wherein said re-training comprises at least generating machine explanations for the subset of training examples that quantify an influence of different parts of a given one of the training examples on outputs of the machine learning model, comparing the user explanations to the machine explanations, and adjusting a parameter configuration of the machine learning model based on said comparing.