Distance-based learning confidence model

A method for jointly training a classification model and a confidence model, the method comprising:

• receiving, at data processing hardware, a training data set comprising a plurality of training data subsets, each training data subset associated with a different respective class and having a plurality of corresponding training examples that belong to the respective class;
• from two or more training data subsets in the training data set:
• selecting, by the data processing hardware, a support set of training examples, the support set of training examples comprising K number of training examples sampled from each of the two or more training data subsets;
• and
• selecting, by the data processing hardware, a query set of training examples, the query set of training examples comprising training examples sampled from each of the two or more training data subsets that are not included in the support set of training examples;
• for each respective class associated with the two or more training data subsets, determining, by the data processing hardware, using the classification model, a centroid value by averaging K number of support encodings associated with the K number of training examples in the support set of training examples that belong to the respective class;
• for each training example in the query set of training examples:
• generating, by the data processing hardware, using the classification model, a query encoding;
• determining, by the data processing hardware, a class distance measure representing a respective distance between the query encoding and the centroid value determined for each respective class;
• determining, by the data processing hardware, a ground-truth distance between the query encoding and a ground-truth label associated with the corresponding training example in the query set of training examples;
• and
• updating, by the data processing hardware, parameters of the classification model based on the class distance measure and the ground-truth distance;
• and
• for each training example in the query set of training examples identified as being misclassified:
• generating, by the data processing hardware, using the confidence model, a standard deviation value for the query encoding generated by the classification model for the corresponding misclassified training example;
• sampling, by the data processing hardware, using the standard deviation value and the query encoding, a new query encoding for the corresponding misclassified training example;
• and
• updating, by the data processing hardware, parameters of the confidence model based on the new query encoding.