Back to results
US 9,002,100 B2Granted
Model uncertainty visualization for active learning
AssigneeXerox Corporation
InventorLoïc M. Lecerf
Life of this patent
Estimating the term…
Abstract
An active learning system and method are disclosed for generating a visual representation of a set of unlabeled elements to be labeled according to class. The representation shows the unlabeled elements as data points in a space and each class as a class point in the space. The position of each of the data points in the space reflects the uncertainty of a model regarding the classification of the respective element. The color of each data point also reflects the uncertainty of the model regarding the classification of the element and may be a mixture of the colors used for the class points.
Claims23
An active learning system comprising:
- a model comprising a probabilistic classifier, which is progressively trained, using labels applied by an annotator to unlabeled elements in a set of elements to be labeled, to associate features of the elements with classes and to predict an appropriate class for an unlabeled element in the set of elements to be labeled, the probabilistic classifier being updated at each of a plurality of iterations to generate a current model with a current probabilistic classifier, whereby at each of the plurality of iterations, the current model assigns to unlabeled elements in the set of elements to be labeled, probabilities that the elements each should be labeled with class labels for a plurality of the classes;
- a representation module which generates at each of the iterations, a visual representation of a set of the unlabeled elements in the set of elements to be labeled based on the current model, the representation showing the unlabeled elements as data points in a space and showing each of a plurality of classes as a class point in the space, each class point having an associated visual identifier, for each of the iterations, the representation module projecting data points of unlabeled elements to positions in the space which are dependent on the uncertainty of the current model regarding a classification of the respective unlabeled elements, a visual identifier of each of the data points being dependent on the visual identifier of at least two of the class points and the uncertainty of the current model regarding the classification of the element;
- a graphical user interface which displays the representation to the annotator and enables the annotator to select one of the unlabeled elements shown in the representation for labeling with a class label for one of the classes;
- an updating module which receives class labels applied by the annotator and updates the model;
- and
- a processor which implements the modules.
3 more elements
An active learning method comprising:
- with a processor, generating a visual representation of a set of elements to be labeled according to class, the representation showing each of a plurality of classes as a class point in a space and showing elements to be labeled as data points in the space, each class point having an associated visual identifier, positions of the data points in the space being dependent on the uncertainty of a model regarding a classification of the respective unlabeled elements, each data point being positioned at a position, relative to at least two of the class points, which has been computed as a function of a probability, assigned by the model, that the respective element should be labeled with that class, a visual identifier of each of the data points being dependent on the visual identifier of at least one of the class points and the uncertainty of the model regarding the classification of the element;
- displaying the representation;
- providing for an annotator to select one of the elements that are represented in the representation for labeling with a class label for one of the classes;
- updating the model based on class labels applied by the annotator;
- and
- generating an updated visual representation in which each data point's position, relative to at least two of the class points, is computed as a function of a probability, assigned by the updated model, that the respective element should be labeled with that class.
3 more elements
A method comprising:
- initializing a model with at least one labeled element for each of a set of classes, the model configured for assigning, to each unlabeled element, probabilities that the element should be labeled with class labels for a plurality of the classes;
- for a set of unlabeled elements to be labeled, generating a visual representation in which each of the classes is represented as a respective class point arranged in the at least two-dimensional space and the unlabeled elements are represented as data points in the space, each class point having a unique color, a position of each of the data points in the space being computed dependent on the assigned probabilities and the data point thereafter being positioned at the computed position, a color of each of the data points being dependent on the colors of the class points and the assigned probabilities;
- and
- with a processor, for each of a plurality of iterations:
- the
- receiving an annotator's class label for at least one of the unlabeled elements;
- updating the model based on the class labels applied by the annotator;
- and
- generating an updated visual representation in which each unlabeled data point's current position, relative to at least two of the class points, is computed as a function of a current probability, assigned by the updated model, that the respective element should be labeled with that class and the data point is thereafter positioned at the computed position.
6 more elements
Dependencies are inferred from the claim text.