Training machine learning models using unsupervised data augmentation

One or more non-transitory computer-readable storage media storing instructions that when executed by one or more computers cause the one more computers to perform -operations comprising:

• receiving training data for training a machine learning model to map model inputs to model outputs in order to perform a particular machine learning task, the training data comprising:
• a plurality of unlabeled training inputs;
• and
• a plurality of labeled training inputs and, for each labeled training input, a ground truth output that should be generated by the machine learning model by performing the particular machine learning task on the labeled training input;
• generating augmented training data, comprising generating, for each of the plurality of unlabeled training inputs, a respective augmented training input by applying a data augmentation technique to the unlabeled training input;
• training the machine learning model on the augmented training data, comprising:
• training the machine learning model on the unlabeled training inputs and the augmented training inputs to optimize an unsupervised objective that measures a difference between (i) a model output generated by the machine learning model for a given unlabeled training input and (ii) a model output generated by the machine learning model for the augmented training input generated from the unlabeled training input, and
• training the machine learning model on the labeled training inputs to optimize a supervised objective that measures a difference between (i) a model output generated by the machine learning model for a given labeled training input and (ii) the ground truth output for the given labeled training input.

A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one more computers to perform operations comprising:

• receiving training data for training a machine learning model to map model inputs to model outputs in order to perform a particular machine learning task, the training data comprising:
• a plurality of unlabeled training inputs;
• and
• a plurality of labeled training inputs and, for each labeled training input, a ground truth output that should be generated by the machine learning model by performing the particular machine learning task on the labeled training input;
• generating augmented training data, comprising generating, for each of the plurality of unlabeled training inputs, a respective augmented training input by applying a data augmentation technique to the unlabeled training input;
• training the machine learning model on the augmented training data, comprising:
• training the machine learning model on the unlabeled training inputs and the augmented training inputs to optimize an unsupervised objective that measures a difference between (i) a model output generated by the machine learning model for a given unlabeled training input and (ii) a model output generated by the machine learning model for the augmented training input generated from the unlabeled training input, and training the machine learning model on the labeled training inputs to optimize a supervised objective that measures a difference between (i) a model output generated by the machine learning model for a given labeled training input and (ii) the ground truth output for the given labeled training input.