Back to results
US 12,748,981 B2Granted
Method and apparatus for transfer learning using sample-based regularization
AssigneeSK Telecom Co., Ltd.
InventorsYong-seok Choi, Yun Ho Jeon, Ji Won Kim, Jae Sun Park, Su Bin Yi, Dong Yeon Cho
Life of this patent
Estimating the term…
Abstract
In training a target model initialized by borrowing the structure and parameters of a pre-trained source model, the present disclosure provides a transfer learning apparatus and method capable of improving the performance of the target model by fine-tuning the target model using sample-based regularization that increases the similarity between features extracted from training samples belonging to the same class.
Claims10
A method of transfer learning from a source model to a target model of a transfer learning apparatus, the method comprising:
- extracting features from input samples using the target model and generating output results of classifying the input samples into classes using the features, wherein the target model comprises a feature extractor extracting the features and a classifier generating the output results;
- calculating a classification loss representing differences between the output results and labels corresponding to the input samples;
- calculating a Sample-Based Regularization (SBR) loss by at least determining a distance metric between feature vectors of a pair of features extracted from a pair of the input samples belonging to same class, wherein the SBR loss regularizes similarity between features within the same class related to a target task of the target model;
- and
- repeatedly updating parameters of the target model based at least on the classification loss to reduce the differences between the output results and the labels, and the SBR loss to increase similarity of an updated pair of features extracted from the pair of the input samples belonging to the same class until the classification loss or the SBR loss converges to a predetermined threshold.
2 more elements
A transfer learning apparatus comprising a target model that is generated by transfer learning from a source model,
- wherein the target model comprises:
- a feature extractor configured to extract features from input samples;
- and
- a classifier configured to generate output results of classifying the input samples into classes using the features,
- wherein the target model is trained by:
- calculating a classification loss representing differences between the output results and labels corresponding to the input samples;
- calculating a Sample-Based Regularization (SBR) loss by at least determining a distance metric between feature vectors of a pair of features extracted from a pair of the input samples belonging to same class, wherein the SBR loss regularizes similarity between features within the same class related to a target task of the target model;
- and
- repeatedly updating parameters of the target model based at least on the classification loss to reduce the differences between the output results and the labels, and the SBR loss to increase similarity of an updated pair of features extracted from the pair of the input samples belonging to the same class until the classification loss or the SBR loss converges to a predetermined threshold.
6 more elements
A classification apparatus generating output results of classifying input samples into classes based on a target model, the classification apparatus comprising:
- a feature extractor extracting features from the input samples;
- and
- a classifier classifying the input samples into the classes based on the features,
- wherein the target model is pre-trained by calculating a classification loss representing differences between the output results and labels corresponding to the input samples;
- calculating a Sample-Based Regularization (SBR) loss by at least determining a distance metric between feature vectors of a pair of features extracted from a pair of the input samples belonging to same class, wherein the SBR loss regularizes similarity between features within the same class related to a target task of the target model;
- and
- repeatedly updating parameters of the target model based at least on the classification loss to reduce the differences between the output results and the labels, and the SBR loss to increase similarity of an updated pair of features extracted from the pair of the input samples belonging to the same class until the classification loss or the SBR loss converges to a predetermined threshold.
4 more elements
A computer-readable recording medium storing instructions that, when being executed by a computer, cause the computer to perform:
- extracting features from input samples using a target model and generating output results of classifying the input samples into classes using the features, wherein the target model comprises a feature extractor extracting the features and a classifier generating the output results;
- calculating a classification loss representing differences between the output results and labels corresponding to the input samples;
- calculating a Sample-Based Regularization (SBR) loss by at least determining a distance metric between feature vectors of a pair of features extracted from a pair of the input samples belonging to same class, wherein the SBR loss regularizes similarity between features within the same class related to a target task of the target model;
- and
- repeatedly updating parameters of the target model based at least on the classification loss to reduce the differences between the output results and the labels, and the SBR loss to increase similarity of an updated pair of features extracted from the pair of the input samples belonging to the same class until the classification loss or the SBR loss converges to a predetermined threshold.
2 more elements
Dependencies are inferred from the claim text.