Model training method and apparatus for federated learning, device, and storage medium

A model training method for federated learning, performed by an ith node device in a vertical federated learning system, comprising n node devices, n being an integer greater than or equal to 2, i being a positive integer less than or equal to n, the method comprising:

• determining an ith second-order gradient descent direction of an ith sub-model:
• generating an ith learning rate operator based on an ith first-order gradient and an ith second-order gradient of the ith sub-model, the ith learning rate operator being used for determining a learning rate in response to performing model training based on the ith second-order gradient descent direction;
• generating, when the ith node device is a first node device, a first sum learning rate operator by:
• generating a random number;
• generating a first learning rate operator and performing a rounding operation on the first learning rate operator;
• determining a first intermediate sum learning rate operator based on at least the first learning rate operator after the rounding operation and the random number;
• and
• performing a modulo operation on the first intermediate sum learning rate operator to obtain the first sum learning rate operator;
• and
• transmitting the first sum learning rate operator to a next node device;
• generating, when the ith node device is not the first node device, an ith sum learning rate operator by:
• receiving an (i−1)th sum learning rate operator transmitted by an (i−1)th node device;
• performing a rounding operation on the ith learning rate operator;
• determining an ith intermediate sum learning rate operator based on at least the ith learning rate operator after the rounding operation and the (i−1)th sum learning rate operator;
• and
• performing a modulo operation on the ith intermediate sum learning rate operator to obtain the ith sum learning rate operator;
• and
• transmitting the ith sum learning rate operator to the next node device,
• wherein updating the ith sub-model based on the ith second-order gradient descent direction comprising:
• updating an ith model parameter of the ith sub-model based on the ith second-order gradient descent direction and the learning rate,
• wherein before determining the ith second-order gradient descent direction of the ith sub-model at the first node device, the method further comprises:
• receiving an nth sum operator transmitted by an nth node device;
• restoring an accumulation result of a first scalar operator to an nth scalar operator based on the random number and the nth sum operator;
• and
• determining a second-order gradient scalar based on the accumulation result.

A computer device, comprising:

• a memory, configured to store at least one program;
• and
• at least one processor, electrically coupled to the memory and configured to execute the at least one program to perform steps comprising:
• generating, by an ith node device in a vertical federated learning system having n node devices, an ith scalar operator based on a (t−1)th round of training data and a tth round of training data, the (t−1)th round of training data comprising an ith model parameter and an ith first-order gradient of an ith sub-model after the (t−1)th round of training, the tth round of training data comprising the ith model parameter and the ith first-order gradient of the ith sub-model after the tth round of training, t being an integer greater than 1, n being an integer greater than or equal to 2, and i being a positive integer less than or equal to n;
• generating, when the ith node device is a first node device, a first sum operator by:
• generating a first scalar operator and performing a round operation on the first scalar operation;
• generating a random number;
• and
• generating the first sum operator based on at least the random number and a first scalar operator, the random number being secret to other node devices;
• and
• transmitting the first sum operator to a next node device;
• generating, when the ith node device is not the first node device, an ith sum operator by:
• receiving an (i−1)th sum operator transmitted by an (i−1)th node device;
• performing a rounding operation on the ith scalar operator;
• determining an ith intermediate sum operator based on at least the ith scalar operator after the rounding operation and the (i−1)th sum operator;
• performing a modulo operation on the ith intermediate sum operator to obtain the ith sum operator;
• transmitting the ith sum operator to the next node device;
• determining an ith second-order gradient descent direction of the ith sub-model based on a second-order gradient scalar, the ith model parameter, and the ith first-order gradient, the second-order gradient scalar being obtained by the first node device based on an nth sum operator;
• and
• updating the ith sub-model based on the ith second-order gradient descent direction to obtain model parameters of the ith sub-model during a (t+1)th round of iterative training,
• wherein before determining the ith second-order gradient descent direction of the ith sub-model at the first node device, the at least one processor is further configured to perform:
• receiving the nth sum operator transmitted by an nth node device;
• restoring an accumulation result of the first scalar operator to an nth scalar operator based on the random number and the nth sum operator;
• and
• determining the second-order gradient scalar based on the accumulation result.