Back to results
US 11,113,285 B2Granted
Partition aware partial query processing
AssigneeSap SE
InventorsDavide Corradi, Johannes Merx, Julian Schwing, Christoph Weyerhaeuser
Life of this patent
Estimating the term…
Abstract
Described herein includes a calculation scenario of a calculation engine that efficiently partitions data for processing at separate hosts, including in parallel, and unions intermediate results from such separate processing when required for further processing. Such parallel processing of partitions can allow for faster processing times, and such unioning of data only when required for further processing can limit the transferring of data that results in slower processing.
Claims20
A system comprising:
- at least one programmable processor;
- and
- machine-readable memory storing instructions, which when executed by the at least one processor cause the at least one programmable processor to perform operations comprising:
- detecting a partitioning flag and at least one set of partition attributes at a first datasource operator in a calculation scenario to be executed in a calculation engine of a database management system, the partitioning flag providing instructions to the calculation engine regarding at least one execution scope, the at least one execution scope including a table;
- partitioning the table into at least one partition based on the at least one set of partition attributes, the partitioning comprising determining, based on the execution scope, to partition the table into a plurality of logical partitions and a plurality of physical partitions;
- determining, based upon the detected partitioning flag, that the table is partitioned into a plurality of partitions, each partition of the plurality of partitions having at least one row corresponding to a same set of values of the at least one set of partition attributes, wherein the plurality of partitions comprises the plurality of logical partitions and the plurality of physical partitions;
- assigning, based upon the detected partitioning flag, a first physical partition of the plurality of physical partitions to a first execution scope of the at least one execution scope, a second physical partition of the plurality of physical partitions to a second execution scope of the at least one execution scope, the first execution scope to a first index server, and the second execution scope to a second index server, the second index server being separate from the first index server, each of the first index server and the second index server running on a computing system having a programmable processor;
- assigning, based upon the detected partitioning flag, a logical partition of the plurality of logical partitions to a third execution scope, and the third execution scope to the first index server;
- duplicating at least one operator in the calculation scenario that follows the partitioning flag to form a plurality of copies of the at least one operator such that one copy of the at least one operator is provided to each of the at least one execution scope for execution on respective index servers to generate a plurality of intermediate results for each execution scope;
- generating the plurality of intermediate results for each execution scope;
- detecting, during the generation of the plurality of intermediate results for each execution scope, a second flag indicating a union of the intermediate results and an end of the at least one partition of the plurality of partitions, the second flag comprising instructions to union the plurality of intermediate results;
- and
- unioning, based on the detection of the second flag, the plurality of intermediate results for further processing.
10 more elements
A computer-implemented method, comprising:
- detecting a partitioning flag and at least one set of partition attributes at a first datasource operator in a calculation scenario to be executed in a calculation engine of a database management system, the partitioning flag providing instructions to the calculation engine regarding at least one execution scope, the execution scope including a table;
- partitioning the table into at least one partition based on the at least one set of partition attributes, the partitioning comprising determining, based on the execution scope, to partition the table into a plurality of logical partitions and a plurality of physical partitions;
- determining, based upon the detected partitioning flag, that the table is partitioned into a plurality of partitions, each partition of the plurality of partitions having at least one row corresponding to a same set of values of the at least one set of partition attributes, wherein the plurality of partitions comprises the plurality of logical partitions and the plurality of physical partitions;
- assigning, based upon the detected partitioning flag, a first physical partition of the plurality of physical partitions to a first execution scope of the at least one execution scope, a second physical partition of the plurality of physical partitions to a second execution scope of the at least one execution scope, the first execution scope to a first index server, and the second execution scope to a second index server, the second index server being separate from the first index server, each of the first index server and the second index server running on a computing system having a programmable processor;
- assigning, based upon the detected partitioning flag, a logical partition of the plurality of logical partitions to a third execution scope, and the third execution scope to the first index server;
- duplicating at least one operator in the calculation scenario that follows the partitioning flag to form a plurality of copies of the at least one operator such that one copy of the at least one operator is provided to each of the at least one execution scope for execution on respective index servers to generate a plurality of intermediate results for each execution scope;
- generating the plurality of intermediate results for each execution scope;
- detecting, during the generation of the plurality of intermediate results for each execution scope, a second flag indicating a union of the intermediate results and an end of the at least one partition of the plurality of partitions, the second flag comprising instructions to union the plurality of intermediate results;
- and
- unioning, based on the detection of the second flag, the plurality of intermediate results for further processing.
7 more elements
A non-transitory computer-readable medium storing instructions, which when executed by at least one data processor, result in operations comprising:
- detecting a partitioning flag and at least one set of partition attributes at a first datasource operator in a calculation scenario to be executed in a calculation engine of a database management system, the partitioning flag providing instructions to the calculation engine regarding at least one execution scope, the execution scope including a table;
- partitioning the table into at least one partition based on the at least one set of partition attributes, the partitioning comprising determining, based on the execution scope, to partition the table into a plurality of logical partitions and a plurality of physical partitions;
- determining, based upon the detected partitioning flag, that the table is partitioned into a plurality of partitions, each partition of the plurality of partitions having at least one row corresponding to a same set of values of the at least one set of partition attributes, wherein the plurality of partitions comprises the plurality of logical partitions and the plurality of physical partitions;
- assigning, based upon the detected partitioning flag, a first physical partition of the plurality of physical partitions to a first execution scope of the at least one execution scope, a second physical partition of the plurality of physical partitions to a second execution scope of the at least one execution scope, the first execution scope to a first index server, and the second execution scope to a second index server, the second index server being separate from the first index server, each of the first index server and the second index server running on a computing system having a programmable processor;
- assigning, based upon the detected partitioning flag, a logical partition of the plurality of logical partitions to a third execution scope, and the third execution scope to the first index server;
- duplicating at least one operator in the calculation scenario that follows the partitioning flag to form a plurality of copies of the at least one operator such that one copy of the at least one operator is provided to each of the at least one execution scope for execution on respective index servers to generate a plurality of intermediate results for each execution scope;
- generating the plurality of intermediate results for each execution scope;
- detecting, during the generation of the plurality of intermediate results for each execution scope, a second flag indicating a union of the intermediate results and an end of the at least one partition of the plurality of partitions, the second flag comprising instructions to union the plurality of intermediate results;
- and
- unioning, based on the detection of the second flag, the plurality of intermediate results for further processing.
7 more elements
Dependencies are inferred from the claim text.