Framework for learning to transfer learn

A method comprising:

• obtaining, by a computing system, a source dataset comprising source images of a first class;
• obtaining, by the computing system, a target dataset comprising target images of a second class, the second class different than the first class;
• and
• at each respective learning iteration of a plurality of learning iterations for a learning to transfer learn (L2TL) architecture:
• training, by the computing system, a deep learning model on the source dataset and the target dataset using a loss function for transfer learning, the loss function comprising:
• a source dataset classification loss based on source classifier layer weights and encoder weights;
• a target dataset classification loss based on target classifier layer weights and the encoder weights;
• a source scaling coefficient;
• a target scaling coefficient;
• and
• a policy weight associated with a policy model;
• while training the deep learning model:
• sampling, by the computing system, a training batch of the source images from the source dataset;
• sampling, by the computing system, a training batch of the target images from the target dataset;
• applying, by the computing system, the policy model to the training batch of source images to assign a respective importance weight to each source image from the training batch of source images;
• determining, by the computing system, and based on each respective importance weight assigned to each source image, the source dataset classification loss for the respective learning iteration;
• determining, by the computing system, and based on each target image from the training batch of target images, the target dataset classification loss for the respective learning iteration;
• determining, by the computing system, the source scaling coefficient and the target scaling coefficient for the respective learning iteration by using the policy model to sample an action;
• determining, by the computing system, based on the loss function, and based on the source dataset classification loss, the target dataset classification loss, the source scaling coefficient, and the target scaling coefficient for the respective learning iteration, a loss for the respective learning iteration;
• and
• applying, by the computing system, and using the loss for the respective learning iteration, gradient descent-based optimization to update the source classifier layer weights, the target classifier layer weights, and the encoder weights that minimize the loss function;
• and
• updating, by the computing system, the policy weight associated with the policy model to maximize an evaluation metric indicative of a performance of the deep learning model on a target evaluation dataset.