Optimization of data repartitioning during parallel query optimization
AssigneeInternational Business Machines Corporation
InventorsGuy Maring Lohman, Mir Hamid Pirahesh, Eugene Jon Shekita, David E. Simmen, Monica Sachiye Urata
Life of this patent
Estimating the term…
Abstract
Query evaluation is optimized using parallel optimization techniques to make repartitioning more efficient. Efficiency is improved by recognizing the possible partitioning requirements for achieving parallelism for a query operation, and by recognizing when the partitioning property of data satisfies the partitioning requirements of a query operation. A data base management system in accordance with the invention uses parallel query processing techniques to optimize data repartitioning, or to avoid it altogether.
Claims44
A method of processing a query in a relational database management system that operates in a computer network to retrieve data from tables in computer storage, the method comprising the steps of:
- receiving two partitioned data streams that relate to a join operation;
- determining whether conjunct predicates can be used to locally perform a parallel inner join or outer join;
- and
- performing the join operation locally if the step of determining indicates appropriate conjunct predicates can be used.
A method of processing a query in a relational database management system that operates in a computer network to retrieve data from tables in computer storage, the method comprising the steps of:
- receiving two partitioned data streams that comprise a source data stream and a target data stream, wherein the data streams relate to a join operation;
- determining whether conjunct predicates can be used to direct tuples of the source data stream to the target data stream;
- and
- performing the join operation after directing the tuples of the source data stream to the target data stream if the step of determining indicates appropriate conjunct predicates can be used.
A method of processing a query in a relational database management system that operates in a computer network to retrieve data from tables in computer storage, the method comprising the steps of:
- receiving two data streams that comprise a source data stream and a target data stream, wherein a subquery predicate will be applied to the data streams;
- determining whether the subquery predicate can be applied to the target data stream locally in parallel;
- and
- applying the determined subquery predicate in parallel, locally, if the step of determining indicates the subquery predicate can be so applied.
A method of processing a query in a relational database management system that operates in a computer network to retrieve data from tables in computer storage, the method comprising the steps of:
- receiving two partitioned data streams that comprise a source data stream and a target data stream, wherein a subquery predicate will be applied to the data streams;
- determining whether the subquery predicate and a conjunct predicate can be used to direct tuples of the source data stream to the target data stream;
- and
- directing the tuples of the source data stream to the target data stream if the step of determining indicates appropriate predicates can be so used.
A method of processing a query in a relational database management system that operates in a computer network to retrieve data from tables in computer storage, the method comprising the steps of:
- receiving a query that includes a relational operator that specifies one or more columns of tables in the computer storage that are distributed with a source partitioning across nodes of the computer network that are to be compared with one or more rows of tables in the computer storage that are distributed with a target partitioning across nodes of the computer network;
- receiving a target partitioning requirement, tables of the source stream of data, and a set of predicates for mapping a partitioning key of the target partitioning requirement to columns of the source stream;
- performing a repartitioning correlated RP operator of the system in response to a request from a consumer node to deliver a repartitioned table to a target node, wherein the RP operator comprises the steps of:
- applying the source partitioning as a predicate operation to a tuple that the RP operator is to send back to the consumer node;
- applying a partitioning function to column values if the RP operator is specified with the target partitioning requirement;
- sending the tuple back to the consumer node if the target node of the RP operator is the same as that of the consumer node;
- and otherwise not sending the tuple back to the consumer node.
A method of processing a query in a relational database management system that operates in a computer network to retrieve data from tables in computer storage, the method comprising the steps of:
- receiving a partitioned data stream to which a table operation comprising either an aggregation operation or a distinct operation will be applied;
- determining whether conjunct predicates can be used to locally perform the table operation in parallel;
- and
- performing the parallel table operation locally if the step of determining indicates appropriate conjunct predicates can be used.
Dependencies are inferred from the claim text.