Method and apparatus for routing data in an inter-nodal communications lattice of a massively parallel computer system by dynamically adjusting local routing strategies

A parallel computer system, comprising:

• a plurality of nodes, each node having at least one processor for executing a respective application sub-process and a memory accessible by the at least one processor;
• a plurality of node-to-node communication links forming a process network for inter-nodal communications of application data;
• a respective routing mechanism in each node of said plurality of nodes, each said routing mechanism automatically routing each of a plurality of packets from a respective source node to a respective destination node along said node-to-node communications links of said network using a respective automated local routing strategy of the respective node, wherein at least some of said packets must traverse one or more respective intermediate nodes between the respective source node and the respective destination node, each said respective local automated routing strategy comprising a respective set of one or more routing rules used by the corresponding said routing mechanism to automatically select a respective routing for each packet routed by the corresponding said node;
• wherein, in at least some said nodes, the respective routing mechanism alters the respective automated local routing strategy of the node during execution of a distributed application in the at least some nodes by altering at least one routing rule of the set of routing rules used by the respective routing mechanism, and thereafter routes each of a plurality of packets from a respective source node to a respective destination node along said node-to-node communications links of said network using a respective local automated routing strategy as altered, wherein said altering comprises the steps of:
• periodically obtaining network usage data with respect to usage of said network of node-to-node communications links;
• analyzing most recently obtained said network usage data to identify a respective optimum local automated routing strategy for each node of said at least some nodes;
• and
• communicating results of said analyzing step to each node of said at least some of nodes for implementation as an altered routing strategy.

A program product for inter-nodal packet routing in a parallel computer system, said parallel computer system comprising a plurality of nodes arranged in a network of node-to-node communications links for inter-nodal communications, each node comprising at least one processor for executing a respective application sub-process and a nodal memory, the program product comprising:

• a plurality of computer executable instructions recorded on signal-bearing media, wherein said instructions, when executed by at least one node of said computer system, cause the at least one node to perform the steps of:
• at each node of said at least one node, automatically routing each of a plurality of packets from a respective source node to a respective destination node along said node-to-node communications links of said network using a respective local automated routing strategy of the respective node, wherein at least some of said packets must traverse one or more respective intermediate nodes between the respective source node and the respective destination node, each said respective local automated routing strategy comprising a respective set of one or more routing rules used by the corresponding said node to automatically select a respective routing for each packet routed by the corresponding said node;
• at each node of said at least one node, altering said respective local automated routing strategy during execution of said at least one application by altering at least one routing rule of the set of routing rules corresponding to the respective node, wherein said altering comprises the steps of:
• periodically obtaining network usage data with respect to usage of said network of node-to-node communications links;
• analyzing most recently obtained said network usage data to identify a respective optimum local automated routing strategy for each node of said at least some nodes;
• and
• communicating results of said analyzing step to each node of said at least some of nodes for implementation as an altered routing strategy;
• and
• responsive to said altering step, at each node of said at least one node, thereafter routing each of a plurality of packets from a respective source node to a respective destination node along said node-to-node communications links of said network using a respective local automated routing strategy as altered by said altering step.