Model training using differential privacy and knowledge transfer
AssigneeSap SE
InventorCaelin KAPLAN
Life of this patent
Estimating the term…
Abstract
Systems and methods include acquisition of training data comprising a plurality of target variable categories, training of a first classification model based on the training data, determination of node weights of the trained first classification model, and training of a second classification model using differential privacy and a first loss function including a weight loss term comparing the determined node weights of the trained first classification model to node weights of the second classification model.
Claims19
A system comprising:
- a memory storing processor-executable program code;
- and
- at least one processing unit to execute the processor-executable program code to cause the system to:
- acquire training data comprising a plurality of target variable categories;
- train a first classification model based on the training data;
- determine node weights of the trained first classification model;
- and
- train a second classification model using differential privacy and a first loss function including a weight loss term comparing the determined node weights of the trained first classification model to node weights of the second classification model.
A method comprising:
- acquiring training data comprising a plurality of instances, each instance comprising a value of each of a plurality of input variables and of a target variable, where the values of the target variable comprise a plurality of categories;
- training a first classification model based on the training data;
- determining node weights of the trained first classification model;
- and
- training a second classification model using a first loss function including a weight loss term comparing the determined node weights of the trained first classification model to node weights of the second classification model, and by:
- for each of a first plurality of the plurality of instances, determining a gradient of the first loss function with respect to the node weights of the second classification model;
- determining a composite gradient of the first loss function with respect to the node weights of the second classification model based on the gradient of the first loss function determined for each of the first plurality of the plurality of instances;
- and
- updating the node weights of the second classification model based on the composite gradient.
A non-transitory medium storing executable program code executable by at least one processing unit of a computing system to cause the computing system to:
- acquire training data comprising a plurality of target variable categories;
- train a first classification model based on the training data;
- determine node weights of the trained first classification model;
- and
- train a second classification model using differential privacy and a first loss function including a weight loss term determining a distance between the determined node weights of the trained first classification model and node weights of the second classification model.
Dependencies are inferred from the claim text.