# US 2021/0240532 A1 · Efficient inter-chip interconnect topology for distributed parallel deep learning > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0240532 A1 - Document ID: US20210240532A1 - Status: Published application - Kind: A1 - Application number: 16/777,683 - Filed: 2020-01-30 - Published: 2021-08-05 - Earliest claimed: 2020-01-30 - Assignees: Alibaba Group Holding Limited - Applicants: Alibaba Group Holding Limited - Inventors: Liang HAN; Yang JIAO - Web page: https://patentgrep.com/patent/US20210240532A1 ## CPC - G06F 9/505: {considering the load} - G06F 9/5044: {considering hardware capabilities} - G06N 3/063: using electronic means - G06N 3/08: Learning methods - G06N 3/098: Distributed learning, e.g. federated learning - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract The present disclosure provides a system comprising: a first group of computing nodes and a second group of computing nodes, wherein the first and second groups are neighboring devices and each of the first and second groups comprising: a set of computing nodes A-D, and a set of intra-group interconnects, wherein the set of intra-group interconnects communicatively couple computing node A with computing nodes B and C and computing node D with computing nodes B and C; and a set of inter-group interconnects, wherein the set of inter-group interconnects communicatively couple computing node A of the first group with computing node A of the second group, computing node B of the first group with computing node B of the second group, computing node C of the first group with computing node C of the second group, and computing node D of the first group with computing node D of the second group. ## Claims ### Claim 1 A system comprising: a first group of computing nodes and a second group of computing nodes, wherein the first and second groups are neighboring devices and each of the first and second groups comprising: a set of computing nodes A-D, and a set of intra-group interconnects, wherein the set of intra-group interconnects communicatively couple computing node A with computing nodes B and C and computing node D with computing nodes B and C; and a set of inter-group interconnects, wherein the set of inter-group interconnects communicatively couple computing node A of the first group with computing node A of the second group, computing node B of the first group with computing node B of the second group, computing node C of the first group with computing node C of the second group, and computing node D of the first group with computing node D of the second group. ### Claim 2 (depends on 1) The system of claim 1, wherein the set of intra-group interconnects and the set of inter-group interconnects comprise inter-chip interconnects. ### Claim 3 (depends on 1) The system of claim 1, further comprising: a third group of computing nodes, wherein the third group of computing nodes is neighboring with the second group and the third group comprising: a set of computing nodes A-D, and a set of intra-group interconnects, wherein the set of intra-group interconnects communicatively couple computing node A with computing nodes B and C and computing node D with computing nodes B and C; and a second set of inter-group interconnects, wherein the second set of inter-group interconnects communicatively couple computing node A of the third group with computing node A of the second group, computing node B of the third group with computing node B of the second group, computing node C of the third group with computing node C of the second group, and computing node D of the third group with computing node D of the second group. ### Claim 4 (depends on 1) The system of claim 1, wherein: the intra-group interconnects and the first set of inter-group interconnects form a ring connection that communicatively couples computing node A of the first group with computing node B of the first group, computing node B of the first group with computing node B of the second group, computing node B of the second group with computing node A of the second group, and computing node A of the second group with computing node A of the first group. ### Claim 5 (depends on 3) The system of claim 3, wherein: the intra-group interconnects and the first and second sets of inter-group interconnects form a ring connection that communicatively couples computing node A of the first group with computing node B of the first group, computing node B of the first group with computing node B of the second group, computing node B of the second group with computing node B of the third group, computing node B of the third group with computing node A of the third group, computing node A of the third group with computing node A of the second group, and computing node A of the second group with computing node A of the first group. ### Claim 6 (depends on 1) The system of claim 1, further comprising: a plurality of groups of computing nodes, including the first group and the second group, wherein: the plurality of groups are grouped into a plurality of sub-sections, wherein at least one sub-section comprises a ring connection that comprises a row or column of computing nodes. ### Claim 7 (depends on 1) The system of claim 1, wherein the computing nodes are processors. ### Claim 8 (depends on 7) The system of claim 7, wherein the computing nodes are artificial intelligence (“AI”) training processors, AI training chips, neural processing units (“NPU”), or graphic processing units (“GPU”). ### Claim 9 (depends on 1) The system of claim 1, wherein the inter-chip interconnects of the set of intra-group interconnects or the set of inter-group interconnects are bi-directional. ### Claim 10 (depends on 9) The system of claim 9, wherein the inter-chip interconnects are copper cables. ### Claim 11 (depends on 1) The system of claim 1, wherein each of first group and the second group are boards, and each board further comprises: a main processor; and a hardware bridge, wherein the hardware bridge connects computing nodes A-D on the board with the main processor. ### Claim 12 A method for allocating a computing task across a plurality of groups of computing nodes, each group of the plurality of groups comprising a set of computing nodes A-D and a set of intra-group interconnects that communicatively couple computing node A with computing nodes B and C and computing node D with computing nodes B and C, the method comprising: acquiring the computing task; determining a hardware load of the computing task; and allocating the computing task to first and second groups of the plurality of groups according to the hardware load, wherein performance of the computing task generates data packets that are forwarded over a set of inter-group interconnects between the first and second groups. ### Claim 13 (depends on 12) The method of claim 12, wherein the set of inter-group interconnects communicatively couple computing node A of the first group with computing node A of the second group, computing node B of the first group with computing node B of the second group, computing node C of the first group with computing node C of the second group, and computing node D of the first group with computing node D of the second group. ### Claim 14 (depends on 13) The method of claim 13, wherein the set of intra-group interconnects and the set of inter-group interconnects comprise inter-chip interconnects. ### Claim 15 (depends on 12) The method of claim 12, wherein allocating the computing task to first and second groups of the plurality of groups according to the hardware load further comprises: dividing the computing nodes in the plurality of groups into one or more sub-sections, wherein the computing task is assigned to a sub-section in the one or more sub-sections. ### Claim 16 (depends on 12) The method of claim 12, further comprising: creating a ring connection from intra-group interconnects and inter-group interconnects that communicatively couples computing node A of the first group with computing node B of the first group, computing node B of the first group with computing node B of the second group, computing node B of the second group with computing node A of the second group, and computing node A of the second group with computing node A of the first group. ### Claim 17 (depends on 12) The method of claim 12, wherein the computing nodes are artificial intelligence (“AI”) training processors, AI training chips, neural processing units (“NPU”), or graphic processing units (“GPU”). ### Claim 18 (depends on 17) The method of claim 17, wherein the computing task is an AI computing task. ### Claim 19 (depends on 12) The method of claim 12, wherein the inter-chip interconnects of the set of intra-group interconnects or the set of inter-group interconnects are bi-directional. ### Claim 20 A non-transitory computer readable medium that stores a set of instructions that is executable by one or more processors of an apparatus to cause the apparatus to initiate a method for allocating a computing task across a plurality of groups of computing nodes, each group comprising a set of computing nodes A-D and a set of intra-group interconnects that communicatively couple computing node A with computing nodes B and C and computing node D with computing nodes B and C, the method comprising: acquiring the computing task; determining a hardware load of the computing task; and allocating the computing task to first and second groups of the plurality of groups according to the hardware load, wherein performance of the computing task generates data packets that are forwarded over a set of inter-group interconnects between the first and second groups. ### Claim 21 (depends on 20) The non-transitory computer readable medium of claim 20, wherein the set of inter-group interconnects communicatively couple computing node A of the first group with computing node A of the second group, computing node B of the first group with computing node B of the second group, computing node C of the first group with computing node C of the second group, and computing node D of the first group with computing node D of the second group. ### Claim 22 (depends on 21) The non-transitory computer readable medium of claim 21, wherein the set of intra-group interconnects and the set of inter-group interconnects comprise inter-chip interconnects. ### Claim 23 (depends on 20) The non-transitory computer readable medium of claim 20, wherein the set of instructions that is executable by one or more processors of the apparatus to cause the apparatus to further perform: dividing the computing nodes in the plurality of groups into one or more sub-sections, wherein the computing task is assigned to a sub-section in the one or more sub-sections. ### Claim 24 (depends on 20) The non-transitory computer readable medium of claim 20, the set of instructions that is executable by one or more processors of the apparatus to cause the apparatus to further perform: creating a rings connection from intra-group interconnects and inter-group interconnects that communicatively couple computing node A of the first group with computing node B of the first group, computing node B of the first group with computing node B of the second group, computing node B of the second group with computing node A of the second group, and computing node A of the second group with computing node A of the first group. ### Claim 25 (depends on 20) The non-transitory computer readable medium of claim 20, wherein the computing nodes are artificial intelligence (“AI”) training processors, AI training chips, neural processing units (“NPU”), or graphic processing units (“GPU”). ### Claim 26 (depends on 25) The non-transitory computer readable medium of claim 25, wherein the computing task is an AI computing task. ### Claim 27 (depends on 20) The non-transitory computer readable medium of claim 20, wherein the inter-chip interconnects of the set of intra-group interconnects or the set of inter-group interconnects are bi-directional.