All-to-all message exchange in parallel computing systems

A non-transitory computer readable medium storing a computer program, comprising program code means for causing a compute node to perform, collectively with other compute nodes, to perform a method for all-to-all message exchange between program tasks connected by an interconnection network with a hierarchical network topology including N>1 hierarchy levels ln, n=1 to N, in which a first level l1 includes a plurality of groups of tasks, and each higher level l(n>1) includes at least one group of level l(n-1) groups, the method comprising:

• defining, for each task, a hierarchical identifier which is unique and dependent on the level groups to which each task belongs in respective hierarchy levels of the hierarchical network topology;
• sending a message, at each task in each of successive phases of the exchange, via the interconnection network to a respective destination task whose hierarchical identifier is determined;
• and
• using the hierarchical identifier determined for the destination task to send the program task and the network topology, such that the resulting exchange pattern for the all-to-all message exchange exploits a hierarchical distance in the hierarchical network topology.