Back to results
US 2026/0122127 A1Published application
Distributed Training Method, System, and Apparatus
AssigneeHuawei Cloud Computing Technologies Co., Ltd.
InventorsXiaoyu Zheng, Xibao Pang, Yunwen Lian, Yi Li, Zonghong Dai
Life of this patent
Estimating the term…
Abstract
A distributed training method includes a management node that obtains a network topology, where the network topology includes a connection relationship between a core switch and a computing node in a computing cluster. Then, the management node determines a communication plan between N computing nodes based on the network topology, where the N computing nodes are computing nodes that are in the computing cluster and that are configured to train a target model in a distributed manner.
Claims20
A method comprising:
- obtaining a network topology, wherein the network topology comprises a connection relationship between a first core switch and a computing node in a computing cluster, wherein the computing cluster comprises M groups, and wherein each group of the M groups comprises one or more computing nodes;
- determining, based on the network topology, a communication plan between N computing nodes that are in the computing cluster and that are configured to train a target model in a distributed manner, wherein the communication plan comprises multiple inter-group paths;
- and
- separately instructing, based on the communication plan, at least two of the N computing nodes to use an inter-group path of the multiple inter-group paths for data communication,
- wherein each inter-group path of the multiple inter-group paths comprises:
- two computing nodes that are in the N computing nodes and that belong to different groups;
- and
- a second core switch configured to connect the two computing nodes,
- wherein the inter-group path is configured to transmit data between the two computing nodes,
- wherein an amount of the data meets a preset condition,
- wherein M is an integer greater than 2, and
- wherein N is an integer greater than 2.
9 more elements
A system comprising:
- a computing cluster comprising M groups, wherein each of the M groups comprises one or more computing nodes;
- K core switches configured to connect computing nodes in different groups in the M groups;
- and
- a management node configured to:
- obtain a network topology, wherein the network topology comprises a connection relationship between the K core switches and a first computing node in the computing cluster;
- determine, based on the network topology, a communication plan between N computing nodes that are in the computing cluster and that are configured to train a target model in a distributed manner, wherein the communication plan comprises multiple inter-group paths;
- and
- separately instruct, based on the communication plan, at least two of the N computing nodes to use an inter-group path of the multiple inter-group paths for data communication,
- wherein each inter-path group of the multiple inter-group paths comprises:
- two computing nodes that are in the N computing nodes and that belong to different groups;
- and
- a core switch that is in the K core switches and that is configured to connect the two computing nodes,
- wherein the inter-group path is configured to transmit data between the two computing nodes,
- wherein an amount of the data meets a preset condition, and
- wherein K, M, and N are all integers greater than 2.
12 more elements
Dependencies are inferred from the claim text.