Label inference in split learning defenses
AssigneeLemon Inc.
InventorsShangyu Xie, Jiankai Sun, Xin Yang, Yuanshun Yao, Tianyi Liu, Taiqing Wang
Life of this patent
Estimating the term…
Abstract
Split learning is provided to train a composite neural network (CNN) model that is split into first and second submodels, including receiving a noise-laden backpropagation gradient, training the surrogate submodel by optimizing a gradient distance loss, and computing an updated dummy label using the first submodel and the trained surrogate submodel to infer label information of the second submodel. Noise can be added to a label of the second submodel or a shared backpropagation gradient to protect the label information.
Claims20
A split learning method to train a composite neural network (CNN) model split into a first submodel and a second submodel, the method comprising:
- receiving a noise-laden backpropagation gradient from a second submodel having a label;
- initializing a surrogate submodel and a dummy label of the surrogate submodel;
- training the surrogate submodel by optimizing a gradient distance loss representing a distance between the noise-laden original backpropagation gradient and a surrogate backpropagation gradient;
- and
- computing an updated dummy label using the first submodel and the trained surrogate submodel, the updated dummy label representing a prediction of the label of the second submodel.
A split learning platform comprising:
- a pre-trained first submodel obtained by training a composite neural network (CNN) model that is split into a first submodel and a second submodel, the first submodel having an input feature, and the second submodel having a label corresponding to the input feature;
- and
- a trained surrogate submodel having a dummy label corresponding to the input feature, the trained surrogate submodel being obtained by training another CNN model that is split into the first submodel and a surrogate submodel, and the dummy label of the trained surrogate submodel representing a prediction of the label of the second submodel.
A non-volatile computer-readable medium having computer-executable instructions stored thereon that, upon execution, cause one or more processors to perform operations comprising:
- receiving an original backpropagation gradient from a neural network model having a label in a split learning system;
- initializing a surrogate model having a dummy label;
- training the surrogate model to mimic the neural network model, comprising optimizing a gradient distance loss to converge a surrogate backpropagation gradient from the surrogate submodel to the original backpropagation gradient from the neural network model;
- and
- computing an updated dummy label using the trained surrogate model, the updated dummy label representing a prediction of the label of the neural network model.
Dependencies are inferred from the claim text.