Model training system, model training method, training device, and training node

A model training method, applied to a training device, wherein the training device is any training device in n training devices, a total of M training nodes are deployed in the n training devices, the training device stores a global index list and a dataset corresponding to a training node deployed in the training device, the global index list indicates at least an index of data in the dataset corresponding to the deployed training node, n is a positive integer greater than or equal to 1, and M is a positive integer greater than or equal to n;

• and the method includes:
• in a current round in a round iteration process, shuffling the global index list by using the random number seed, and performing, based on the shuffled list, data query and batch division on the dataset stored in the training device, to obtain exchange data of the training node deployed in the training device in a current batch in a batch iteration process, wherein batch division is based on a sequence of indexes indicated by the shuffled list, the sequence of the indexes indicates a data sequence, the current batch is any batch in a plurality of batches in the current round, shuffled lists in a plurality of rounds are different, and the exchange data is hit data obtained through data query;
• generating, by the at least one processor, a global index mask list corresponding to the shuffled list by performing mask processing on index values that are beyond an index range of the dataset stored in the training device;
• obtaining, by the at least one processor, a global mask data list corresponding to the global index mask list;
• determining, by the at least one processor, the exchange data based on the global mask data list;
• obtaining, by the at least one processor, shared data, and obtaining, by the at least one processor, training data corresponding to the training node deployed in the training device based on the shared data, wherein the shared data is obtained by mutually complementing respective hit data of the M training nodes;
• and
• performing, by the training node deployed in the training device, model training based on the corresponding training data.

A model training method, applied to a first training node, wherein the first training node is any one of M training nodes, the M training nodes are deployed in n training devices, each of the n training devices stores a global index list and a dataset corresponding to a training node deployed in the training device, the global index list indicates at least an index of data in a dataset corresponding to the first training node, n is a positive integer greater than or equal to 1, and M is a positive integer greater than or equal to n;

• and the method includes:
• in a current round of a round iteration process, shuffling, by at least one processor of a training device in which the first training node is deployed, the global index list by using a random number seed, and performing, by the at least one processor and based on a shuffled list, data query and batch division on the dataset corresponding to the first training node, to obtain exchange data of the first training node in a current batch of a batch iteration process, wherein the exchange data is hit data obtained through data query, batch division is based on a sequence of indexes indicated by the shuffled list, the sequence of the indexes indicates a data sequence, the current batch is any batch in a plurality of batches in the current round, and shuffled lists in a plurality of rounds are different;
• generating, by the at least one processor, a global index mask list corresponding to the shuffled list by performing mask processing on index values that are beyond an index range of the dataset stored in the training device in which the first training node is deployed;
• obtaining, by the at least one processor, a global mask data list corresponding to the global index mask list;
• determining, by the at least one processor, the exchange data based on the global mask data list;
• obtaining, by the at least one processor, respective exchange data of other training nodes in the M training nodes, and then performing convergence by the at least one processor, to obtain shared data, wherein the shared data is obtained by mutually complementing respective hit data of the M training nodes;
• and
• obtaining, by the at least one processor, training data of the first training node based on the shared data to perform model training.