Machine learning systems and methods for training with noisy labels
AssigneeD-Wave Systems Inc.
InventorArash Vahdat
Life of this patent
Estimating the term…
Abstract
Machine learning classification models which are robust against label noise are provided. Noise may be modelled explicitly by modelling “label flips”, where incorrect binary labels are “flipped” relative to their ground truth value. Distributions of label flips may be modelled as prior and posterior distributions in a flexible architecture for machine learning systems. An arbitrary classification model may be provided within the system. The classification model is made more robust to label noise by operation of the prior and posterior distributions. Particular prior and approximating posterior distributions are disclosed.
Claims19
A method for instantiating a machine learning system for classifying one or more observed elements of an input dataset, the input dataset labelled with one or more observed labels drawn from a plurality of binary labels, the method executed by at least one processor in communication with at least one memory and comprising:
- instantiating a recognition system in the at least one memory, the recognition system operable to classify elements of the input dataset by generating a classification probability for the at least one observed element;
- generating a prior distribution, the prior distribution modelling label noise as a set of stochastic label flips on the plurality of binary labels, each label flip indicating a belief in the correctness of a corresponding label;
- instantiating an approximating posterior system in the at least one memory, the approximating posterior system being a deep neural network operable to generate a posterior probability for one or more label flips for the observed element given the one or more observed labels;
- modeling an approximating posterior distribution using the approximating posterior system, the approximating posterior distribution approximating a true posterior distribution corresponding to the prior distribution, wherein at least one of the prior and posterior distribution comprises a spike, the spike defining a high-probability state where no labels are flipped;
- and
- training at least one of the recognition system and the approximating posterior system based on a training dataset, the prior distribution, and the approximating posterior distribution.
A method for training a machine learning system for classifying one or more observed elements of an input dataset, the input dataset labelled with one or more observed labels, the method executed by at least one processor in communication with at least one memory and comprising:
- instantiating an inference system in the at least one memory;
- modelling a joint probability distribution over a plurality of variables using the inference system, the plurality of variables comprising the observed labels and one or more true labels, the joint probability distribution conditional on the input dataset;
- instantiating an auxiliary system in the at least one memory;
- modelling an auxiliary probability distribution over the plurality of variables independently of the input dataset using the auxiliary system;
- instantiating a representation system in the at least one memory, the representation system operable to characterize the one or more interactions between the plurality of variables of the joint distribution;
- training at least one of the representation system and the inference system based on a first modified lower bound defined over at least a noisy subset of the input dataset, the noisy subset comprising input data and associated observed labels, the first modified lower bound comprising an original lower bound based on the joint probability distribution and an additional term based on the auxiliary probability distribution.
Dependencies are inferred from the claim text.