Training machine learning model
AssigneeInternational Business Machines Corporation
InventorsShiwan Zhao, Bing Zhe Wu, Zhong Su
Life of this patent
Estimating the term…
Abstract
Techniques are provided for training machine learning model. According to one aspect, a training data is received by one or more processing units. The machine learning model is trained based on the training data, wherein the training comprises: optimizing the machine learning model based on stochastic gradient descent (SGD) by adding a dynamic noise to a gradient of a model parameter of the machine learning model calculated by the SGD.
Claims20
A method for training a machine learning model, comprising:
- acquiring, by one or more processing units, a training data;
- and
- training, by one or more processing units, the machine learning model based on the training data, the training comprising:
- optimizing, by one or more processing units, the machine learning model based on stochastic gradient descent (SGD), and
- minimizing privacy leakage by adding a dynamic noise to a gradient of a model parameter of the machine learning model calculated by the SGD.
A computer system, comprising:
- a processor;
- a non-transitory computer-readable memory coupled to the processor, the memory comprising instructions that when executed by the processor perform actions of:
- acquiring, by one or more processing units, a training data;
- and
- training, by one or more processing units, the machine learning model based on the training data, the training comprising:
- optimizing, by one or more processing units, the machine learning model based on stochastic gradient descent (SGD), and
- minimizing privacy leakage by adding a dynamic noise to a gradient of a model parameter of the machine learning model calculated by the SGD.
A computer program product for training a machine learning model, comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to:
- acquiring, by one or more processing units, a training data;
- training, by one or more processing units, the machine learning model based on the training data, the training comprising:
- optimizing, by one or more processing units, the machine learning model based on stochastic gradient descent (SGD) by adding a dynamic noise, selected from a predefined noise set, to a gradient of a model parameter of the machine learning model calculated by the SGD wherein the optimizing further comprises minimizing a loss function of the machine learning model, and
- assigning a corresponding probability to each of the noises in the predefined noise set according to the loss function, wherein each of the noises is with a different scale from each other.
Dependencies are inferred from the claim text.