Automatic and dynamic adaptation of grouping in a data processing system

A first leader node of a first group of nodes in communication with a plurality of leader nodes in a distributed data processing system, comprising:

• processing circuitry, configured to:
• determine a cost map including computational and/or communication costs within said first group of nodes;
• calculate costs between said first leader node and first member nodes within said first group of nodes based on said cost map;
• determine a candidate member node from said first member nodes if said costs between said first leader node and said candidate member node exceeds a threshold;
• query said plurality of leader nodes to accept a transfer of said candidate member node;
• receive a first transfer response from a second leader node of said plurality of leader nodes to accept said transfer of said candidate member node to a second group of nodes directed by said second leader node;
• initiate a transfer of said candidate member node to said second group of nodes if said first leader node accepts said transfer thereto;
• and
• receive an acknowledgement of said transfer of said candidate member node to said second group of nodes.

A first leader node of a first group of nodes in communication with a second leader node of a second group of nodes in a distributed data processing system, comprising:

• processing circuitry, configured to:
• receive a query from said second leader node to accept a transfer of a candidate member node of second member nodes of said second group of nodes to said first group of nodes;
• provide a transfer response to said second leader node if said first leader node accepts said transfer of said candidate member node to said first group of nodes;
• receive an instruction to transfer said candidate member node to said first group of nodes;
• and
• provide an acknowledgement of said transfer of said candidate member node to said first group of nodes
• wherein said processing circuitry is further configured to:
• calculate costs of said candidate member node within said first group of nodes based on a cost map including computational and/or communication costs within said first group of nodes, and provide said transfer response to accept said transfer if said costs between said first leader node and said candidate member node within said first group of nodes are less than a threshold;
• and
• if said costs between said first leader node and said candidate member node within said first group of nodes are equal to or greater than the threshold, update membership information and status for first member nodes within said first group of nodes.

A method of operating a first leader node of a first group of nodes in communication with a second leader node of a second group of nodes in a distributed data processing system, comprising:

• receiving a query from said second leader node to accept a transfer of a candidate member node of second member nodes of said second group of nodes to said first group of nodes;
• providing a transfer response to said second leader node if said first leader node accepts said transfer of said candidate member node to said first group of nodes;
• receiving an instruction to transfer said candidate member node to said first group of nodes;
• and
• providing an acknowledgement of said transfer of said candidate member node to said first group of nodes, further comprising either;
• calculating costs of said candidate member node within said first group of nodes based on a cost map including computational and/or communication costs within said first group of nodes;
• and
• providing said transfer response to accept said transfer if said costs between said first leader node and said candidate member node within said first group of nodes are less than a threshold and further comprising comparing said costs between said first leader node and said candidate member node within said first group of nodes.