# US 2025/0094397 A1 · Model training system, model training method, training device, and training node

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0094397 A1
- Document ID: US20250094397A1
- Status: Published application
- Kind: A1
- Application number: 18/962,302
- Filed: 2024-11-27
- Published: 2025-03-20
- Earliest claimed: 2022-05-30
- Assignees: Huawei Technologies Co., Ltd.
- Applicants: Huawei Technologies Co., Ltd.
- Inventors: Yifan YAO; Teng SU
- Web page: https://patentgrep.com/patent/US20250094397A1

## CPC

- G06F 16/2228: {Indexing structures}
- G06N 3/045: Combinations of networks
- G06F 9/5027: {the resource being a machine, e.g. CPUs, Servers, Terminals}
- G06F 9/54: Interprogram communication
- G06F 16/245: Query processing
- G06F 16/2465: {Query processing support for facilitating data mining operations in structured databases}
- G06F 16/9024: {Graphs; Linked lists (G06F16/9027 takes precedence)}
- G06N 3/098: Distributed learning, e.g. federated learning
- G06N 20/00: Machine learning
- G06N 20/10: using kernel methods, e.g. support vector machines [SVM]
- G06F 2209/509: Offload
- G06F 2209/544: Remote
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

The present invention provides a model training system, a model training method, a training device, and a training node. The system includes: n training devices, in which a total of M training nodes are deployed. Each training device stores a global index list and a dataset corresponding to a training node deployed in the training device, and the global index list indicates indexes of data in datasets respectively corresponding to the M training nodes. Each training device is configured to: shuffle the global index list by using a random number seed, and query, based on a shuffled list, the dataset stored in the training device, to obtain exchange data of the training node deployed in the training device, where the exchange data is hit data obtained through data query.

## Claims

### Claim 1

A model training system, comprising:
n training devices, in which a total of M training nodes are deployed, wherein each training device stores a global index list and a dataset corresponding to a training node deployed in the training device, the global index list indicates indexes of data in datasets respectively corresponding to the M training nodes, n is a positive integer greater than or equal to 1, M is a positive integer greater than or equal to n, and each of the n devices meets the following descriptions of a first device:
the first device is configured to: shuffle the global index list by using a random number seed, and query, based on a shuffled list, the dataset stored in the training device, to obtain exchange data of the training node deployed in the training device, wherein the exchange data is hit data obtained through data query; and
the first device is further configured to: obtain shared data, and obtain, based on the shared data, training data corresponding to the training node deployed in the training device, wherein the shared data is obtained by mutually complementing respective hit data of the M training nodes, wherein
the M training nodes are configured to perform model training based on corresponding training data.

### Claim 2 (depends on 1)

The model training system according to claim 1, wherein
for any training node in the M training nodes, the training node is configured to: obtain respective exchange data of other training nodes in the M training nodes, to obtain the shared data, and obtain training data of the training node based on the shared data to perform model training.

### Claim 3 (depends on 1)

The model training system according to claim 1, wherein
the first device is configured to: in a current round in a round iteration process, shuffle the global index list by using the random number seed, and perform, based on the shuffled list, data query and batch division on the dataset stored in the first device, to obtain exchange data of the training node deployed in the first device in a current batch in a batch iteration process, wherein batch division is based on a sequence of indexes indicated by the shuffled list, the sequence of the indexes indicates a data sequence, the current batch is any batch in a plurality of batches in the current round, and shuffled lists in a plurality of rounds are different.

### Claim 4 (depends on 3)

The model training system according to claim 3, wherein the hit data is data hit by performing data query on an exchange index list in the current batch, the exchange index list indicates a data index that is in the current batch and that is obtained by performing batch division on the shuffled list, and respective exchange index lists that are in the current batch and that are of the M training nodes are the same; and
the shared data is data obtained by eliminating data that is not hit in the exchange index list and that is of the training node deployed in the first device.

### Claim 5 (depends on 3)

The model training system according to claim 3, wherein the first device is configured to: query, based on the shuffled list, the dataset stored in the first device, to obtain a global mask data list corresponding to the shuffled list; and perform batch division based on the global mask data list, to determine the exchange data of the training node deployed in the first device in the current batch.

### Claim 6 (depends on 5)

The model training system according to claim 5, wherein the first device is configured to: perform mask processing on an index value that is in the shuffled global index list and that is beyond an index range of the dataset stored in the first device, to obtain a global index mask list; and perform, based on the global index mask list, data query on the dataset stored in the first device, to obtain the global mask data list corresponding to the global index mask list.

### Claim 7 (depends on 3)

The model training system according to claim 3, wherein the global index list comprises at least one batch index value corresponding to each of the M training nodes, and the batch index value indicates enhanced batch data that is in one batch and that is in the dataset corresponding to the training node;
the first device is configured to perform, in a unit of batch, data enhancement on the dataset stored in the first device, to obtain enhanced batch data of each of at least one batch index value corresponding to the training node deployed in the first device; and
the current batch corresponds to a target index value, and the target index value is any one or more of the at least one batch index value; and the hit data is enhanced batch data corresponding to the target index value.

### Claim 8 (depends on 7)

The model training system according to claim 7, wherein there are a plurality of batch index values corresponding to the training node deployed in the first device, and a local data index list of the dataset corresponding to the training node deployed in the first device is stored; and
the first device is configured to: shuffle the local data index list stored in the first device and divide a shuffled list, to determine respective data index lists of the plurality of batch index values corresponding to the training node deployed in the first device; and for any batch index value in the plurality of batch index values, perform, based on a data index list of the batch index value, data query on the dataset stored in the first device, and perform data enhancement on found data, to obtain the enhanced batch data.

### Claim 9

A model training method, applied to a training device, wherein the training device is any training device in n training devices, a total of M training nodes are deployed in the n training devices, the training device stores a global index list and a dataset corresponding to a training node deployed in the training device, the global index list indicates at least an index of data in the dataset corresponding to the deployed training node, n is a positive integer greater than or equal to 1, and M is a positive integer greater than or equal to n; and the method includes:
shuffling the global index list by using a random number seed, and performing, based on a shuffled list, data query on the dataset stored in the training device, to obtain exchange data corresponding to the training node deployed in the training device, wherein the exchange data is hit data obtained through data query;
obtaining shared data, and obtaining, based on the shared data, training data corresponding to the training node deployed in the training device, wherein the shared data is obtained by mutually complementing respective hit data of the M training nodes; and
performing, by the training node deployed in the training device, model training based on the corresponding training data.

### Claim 10 (depends on 9)

The method according to claim 9, wherein the obtaining shared data, and obtaining, based on the shared data, training data corresponding to the training node deployed in the training device comprises:
obtaining, by the training node deployed in the training device, exchange data of other training nodes in the M training nodes, and then performing convergence, to obtain the shared data; and obtaining the training data of the training node based on the shared data.

### Claim 11 (depends on 9)

The method according to claim 9, wherein the shuffling the global index list by using a random number seed, and performing, based on a shuffled list, data query on the dataset stored in the training device, to obtain exchange data corresponding to the training node deployed in the training device comprises:
in a current round in a round iteration process, shuffling the global index list by using the random number seed, and performing, based on the shuffled list, data query and batch division on the dataset stored in the training device, to obtain exchange data of the training node deployed in the training device in a current batch in a batch iteration process, wherein batch division is based on a sequence of indexes indicated by the shuffled list, the sequence of the indexes indicates a data sequence, the current batch is any batch in a plurality of batches in the current round, and shuffled lists in a plurality of rounds are different.

### Claim 12 (depends on 11)

The method according to claim 11, wherein the hit data is data hit by performing data query on an exchange index list in the current batch, the exchange index list indicates a data index that is in the current batch and that is obtained by performing batch division on the shuffled list, and respective exchange index lists that are in the current batch and that are of the M training nodes are the same; and
the shared data is data obtained by eliminating data that is not hit in the exchange index list and that is of the training node deployed in the training device.

### Claim 13 (depends on 11)

The method according to claim 11, wherein the shuffling the global index list by using the random number seed, and performing, based on the shuffled list, data query and batch division on the dataset stored in the training device, to obtain exchange data of the training node deployed in the training device in a current batch in a batch iteration process comprises:
querying, based on the shuffled list, the dataset stored in the training device, to obtain a global mask data list corresponding to the shuffled list; and
performing batch division based on the global mask data list, to determine the exchange data in the current batch.

### Claim 14 (depends on 13)

The method according to claim 13, wherein the querying, based on the shuffled list, the dataset stored in the training device, to obtain a global mask data list corresponding to the shuffled list comprises:
performing mask processing on an index value that is in the shuffled list and that is beyond an index range of the dataset stored in the training device, to obtain a global index mask list; and
performing, based on the global index mask list, data query on the dataset stored in the training device, to obtain the global mask data list corresponding to the global index mask list.

### Claim 15 (depends on 11)

The method according to claim 11, wherein the global index list comprises at least one batch index value corresponding to the training node deployed in the training device, and the batch index value indicates enhanced batch data that is in one batch and that is in the dataset stored in the training device;
the current batch corresponds to a target index value, and the target index value is any one or more of the at least one batch index value;
the hit data is enhanced batch data corresponding to the target index value; and
the method further comprises:
performing, in a unit of batch, data enhancement on the dataset stored in the training device, to obtain enhanced batch data of each of the at least one batch index value corresponding to the training node deployed in the training device.

### Claim 16 (depends on 15)

The method according to claim 15, wherein there are a plurality of batch index values of the training node, and the training device stores a local data index list of the dataset corresponding to the deployed training node; and
the method further comprises:
shuffling the local data index list stored in the training device and dividing a shuffled list, to determine respective data index lists of the plurality of batch index values of the training node deployed in the training device; and
for any index value in the plurality of batch index values, performing, based on a data index list of the batch index value, data query on the dataset stored in the training device, and performing data enhancement on found data, to obtain the enhanced batch data.

### Claim 17

A model training method, applied to a first training node, wherein the first training node is any one of M training nodes, the M training nodes are located in n training devices, each of the n training devices stores a global index list and a dataset corresponding to a training node deployed in the training device, the global index list indicates at least an index of data in a dataset corresponding to the first training node, n is a positive integer greater than or equal to 1, and M is a positive integer greater than or equal to n; and the method includes:
receiving exchange data sent by a processor of a training device in which the first training node is located, wherein the exchange data is obtained by the processor by shuffling the global index list by using a random number seed, and performing, based on a shuffled list, data query on the dataset corresponding to the first training node, wherein the exchange data is hit data obtained through data query;
obtaining respective exchange data of other training nodes in the M training nodes, and then performing convergence, to obtain shared data, wherein the shared data is obtained by mutually complementing respective hit data of the M training nodes; and
obtaining, based on the shared data, training data of the first training node to perform model training.

### Claim 18 (depends on 17)

The method according to claim 17, wherein the exchange data is data that is in a current batch and that belongs to the first training node in a next-batch iteration process in one round.

### Claim 19 (depends on 18)

The method according to claim 18, wherein the hit data is data hit by performing data query on an exchange index list in the current batch, the exchange index list indicates a data index that is in the current batch and that is obtained by performing batch division on the shuffled list, and respective exchange index lists that are in the current batch and that are of the M training nodes are the same; and
the shared data is data obtained by eliminating data that is not hit in the exchange index list.

### Claim 20 (depends on 18)

The method according to claim 18, wherein the global index list comprises at least one batch index value corresponding to the first training node, the batch index value indicates enhanced batch data that is in one batch and that is in the dataset, and the enhanced batch data is obtained by the processor by performing, in a unit of batch, data enhancement on the dataset corresponding to the first training node;
the current batch corresponds to a target index value, and the target index value is any one or more of the at least one batch index value; and
the hit data is enhanced batch data corresponding to the target index value.