Logical Data Shuffling
AssigneeInternational Business Machines Corportation
InventorsRonald J. Barber, Robert S. Germain, Guy M. Lohman, Rene Mueller, Ippokratis Pandis, Vijayshankar Raman
Life of this patent
Estimating the term…
Abstract
Embodiments relate to data shuffling by logically rotating processing nodes. The nodes are logically arranged in a two or three dimensional matrix. Every time two of the nodes in adjacent rows of the matrix are positionally aligned, these adjacent nodes exchange data. The positional alignment is a logical alignment of the nodes. The nodes are logically arranged and rotated, and data is exchanged in response to the logical rotation.
Claims20
A method comprising:
- arranging a plurality of computing nodes in a cluster, the arrangement including a coordinated communication topology, and a static distribution of data within the topology;
- pre-determining a step sequence, a step encompassing an alignment of nodes into pairs within the topology, wherein the node pair includes at least one partner node;
- exchanging data among the aligned node pairs, wherein the data exchange supports bi-directional exchange between node pairs;
- and
- re-distributing data among nodes in the topology in response to completion of the step-sequence.
A computer program product for data shuffling, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to perform a method comprising:
- arranging, by the processor, a plurality of computing nodes in a coordinated communication topology, and a static distribution of data within the topology;
- pre-determining a step-sequence, a step encompassing an alignment of nodes into pairs within the topology, wherein the node pair includes at least one node partner;
- exchanging data among the aligned node pairs, wherein the data exchange supports bi-directional exchange between node pairs;
- and
- re-distributing data among nodes in the topology in response to completion of the step-sequence.
A system comprising:
- a processing unit in communication with an application, the application to support data shuffling among a plurality of processing nodes;
- the application functionality comprising:
- a logical arrangement of the processing nodes in a coordinated communication topology and a static distribution of data within the topology;
- a pre-determined step sequence, a step in the sequence encompassing an alignment of nodes into node pairs within the topology, wherein the node pair includes at least one partner node;
- an application to support data exchange among the aligned node pairs, wherein the data exchange supports bi-directional exchange between node pairs;
- and
- the application to distribute data among nodes in the topology in response to completion of the step sequence.
Dependencies are inferred from the claim text.