Efficient inter-chip interconnect topology for distributed parallel deep learning

A method for allocating a computing task across a plurality of groups of computing nodes, each group of the plurality of groups comprising a set of computing nodes A-D and a set of intra-group interconnects that communicatively couple computing node A with computing nodes B and C and computing node D with computing nodes B and C, the method comprising:

• acquiring the computing task;
• determining a hardware load of the computing task;
• and
• allocating the computing task to first and second groups of the plurality of groups according to the hardware load, wherein performance of the computing task generates data packets that are forwarded over a set of inter-group interconnects between the first and second groups.

A non-transitory computer readable medium that stores a set of instructions that is executable by one or more processors of an apparatus to cause the apparatus to initiate a method for allocating a computing task across a plurality of groups of computing nodes, each group comprising a set of computing nodes A-D and a set of intra-group interconnects that communicatively couple computing node A with computing nodes B and C and computing node D with computing nodes B and C, the method comprising:

• acquiring the computing task;
• determining a hardware load of the computing task;
• and
• allocating the computing task to first and second groups of the plurality of groups according to the hardware load, wherein performance of the computing task generates data packets that are forwarded over a set of inter-group interconnects between the first and second groups.