Systems and methods for machine learning enhanced by human measurements
AssigneePresident and Fellows of Harvard College
InventorsDavid Cox, Walter Scheirer, Samuel Anthony, Ken Nakayama
Life of this patent
Estimating the term…
Abstract
In various embodiments, training objects are classified by human annotators, psychometric data characterizing the annotation of the training objects is acquired, a human-weighted loss function based at least in part on the classification data and the psychometric data is computationally derived, and one or more features of a query object are computationally classified based at least in part on the human-weighted loss function.
Claims17
A computer-implemented method for data classification and identification, the method comprising:
- providing, over a computer network, data corresponding to a plurality of training objects to a plurality of training devices each associated with one of a plurality of human annotators, each of the training objects comprising features for classification;
- displaying the training objects on a display of each of the training devices;
- receiving, via communication interfaces of at least some of the training devices, classification data comprising at least some of the training objects annotated, via annotation interfaces of the training devices, by at least some of the annotators with classifications for features thereof;
- acquiring psychometric data characterizing the annotation of the training objects by the annotators;
- computationally deriving a human-weighted loss function based at least in part on the classification data and the psychometric data, the loss function comprising penalties for misclassification, magnitudes of the penalties increasing with increasing deviation from the classification data;
- receiving, by a classification device, data corresponding to a query object different from the plurality of training objects;
- and
- thereafter, computationally classifying, by a computer processor, at least one feature of the query object based at least in part on the human-weighted loss function.
A system for data classification and identification, the system comprising:
- a database of training objects, the database comprising a storage medium populated with stored computer records specifying, for each of a plurality of training objects, (i) classification data comprising annotations received from a plurality of human annotators, and (ii) psychometric data characterizing the annotation of the training object by the plurality of human annotators;
- a computer processor;
- a classification device for receiving query objects different from the training objects in the database;
- a penalization module, executable by the computer processor, for deriving a human-weighted loss function based at least in part on the classification data in the database and the psychometric data of at least some of the training objects in the database, the loss function comprising penalties for misclassification, magnitudes of the penalties increasing with increasing deviation from the classification data;
- and
- a classification module, executable by the computer processor, for classifying features of query objects based at least in part on the human-weighted loss function.
Dependencies are inferred from the claim text.