Data skew insensitive parallel join scheme
AssigneeHewlett-Packard Development Company, L.P.
InventorsAwny K. Al-Omari, QiFan Chen, Gregory S. Battas, Kashif A. Siddiqui, Michael J. Hanlon
Life of this patent
Estimating the term…
Abstract
A method for creating a joined data set from a join input data set is disclosed. The method starts by categorizing the join input data set into a high-skew data set and a low-skew data set. The low-skew data set is distributed to the plurality of CPUs using a first distribution method. The high-skew data set is distributed to the plurality of CPUs using a second distribution method. The plurality of CPUs process the high-skew data set and the low-skew data set to create the joined data set.
Claims20
In a computing system having a plurality of interconnected central processing units (CPUs) and a memory storing a join input data set, a method for creating a joined data set, the steps of the method comprising:
- categorizing the join input data set into a high-skew data set and a low-skew data set during a query optimization phase;
- at query execution time, distributing the low-skew data set to the plurality of CPUs using a first distribution method;
- at query execution time, distributing the high-skew data set to the plurality of CPUs using a second distribution method wherein the first distribution method is different from the second distribution method;
- and
- wherein the plurality of CPUs process the high-skew data set and the low-skew data set to create the joined data set.
A computer software product that includes a medium readable by a processor, the medium having stored thereon, code that when executed by the processor performs a method comprising:
- categorizing the join input data set into a high-skew data set and a low-skew data set during a query optimization phase;
- at query execution time, distributing the low-skew data set to a plurality of CPUs using a first distribution method;
- at query execution time, distributing the high-skew data set to the plurality of CPUs using a second distribution method;
- and
- wherein the plurality of CPUs process the high-skew data set and the low-skew data set to create the joined data set.
Dependencies are inferred from the claim text.