Online hyperparameter tuning in distributed machine learning

A method, comprising:

• sending a first version of a global statistical model, including hyperparameter values and values of other parameters that are not hyperparameters, to a plurality of local model trainers that are each configured to train a local version of the global statistical model;
• wherein the hyperparameter values include regularization values that are each associated with one of the local versions of the global statistical model and correspond to a level of tuning for a respective entity of a plurality of different entities;
• receiving, from the plurality of local model trainers, a plurality of local model performance metrics that correspond to user feedback on outputs produced by the local versions of the global statistical model;
• updating the hyperparameter values based on the received plurality of local model performance metrics;
• including the updated hyperparameter values in a second version of the global statistical model;
• and
• sending the second version of the global statistical model, including the updated hyperparameter values, to the plurality of local model trainers;
• wherein the updated hyperparameter values include updated regularization values that each correspond to a change in the level of tuning for the respective entity.

A method, comprising:

• receiving, from a statistical model merging apparatus, a first version of a global statistical model, including hyperparameter values and values of other parameters that are not hyperparameters;
• wherein the hyperparameter values include regularization values that are each associated with a local version of the global statistical model and correspond to a level of tuning for a specific entity of a plurality of different entities;
• creating the local version of the global statistical model by tuning the first version of the global statistical model for the specific entity;
• sending, to the statistical model merging apparatus, a plurality of local model performance metrics that correspond to user feedback on outputs produced by the local version of the global statistical model;
• and
• receiving, from the statistical model merging apparatus, a second version of the global statistical model, including updated hyperparameter values;
• wherein the updated hyperparameter values include an updated regularization value that corresponds to a change in the level of tuning for the specific entity.

A method, comprising:

• sending, by a statistical model merging apparatus, a first version of a global statistical model, to a plurality of local model trainers that are each configured to train a local version of the global statistical model for a different respective entity;
• receiving, by a local model trainer of the plurality of local model trainers, from the statistical model merging apparatus, a regularization value that is associated with a local version of the global statistical model and corresponds to a level of tuning for a specific entity;
• creating, by the local model trainer, the local version of the global statistical model by tuning the first version of the global statistical model for the specific entity;
• receiving, by the local model trainer, user feedback responsive to output of the local version of the global statistical model;
• computing, by the local model trainer, a plurality of local model performance metrics based on the received user feedback;
• sending, by the local model trainer, to the statistical model merging apparatus, the plurality of local model performance metrics;
• updating, by the statistical model merging apparatus, the regularization value based on the plurality of local model performance metrics;
• including, by the statistical model merging apparatus, the updated regularization value in a second version of the global statistical model;
• and
• sending, by the statistical model merging apparatus, the second version of the global statistical model, including the updated regularization value, to the local model trainer;
• wherein the updated regularization value corresponds to a change in the level of tuning for the specific entity.