Reparallelization for workload skewing database operations

A computer-implemented method, comprising:

• determining whether an operation within a query pipeline is a query is a workload skewing operation, wherein the query pipeline is an architecture in which a query is divided into a sequence of operations for execution of the query, wherein a size of an output dataset of the workload skewing operation is more than a threshold larger or more than a threshold smaller than an input dataset of the workload skewing operation, and wherein the query pipeline comprises one or more first operations following a first scheduling operation that is performed to determine a first task size for the one or more first operations based on a first quantity of time required to perform the one or more first operations on a first portion of the output dataset;
• in response to determining that an operation is a workload skewing operation, inserting, in the query pipeline, a reparallelization point subsequent to the workload skewing operation, wherein the reparallelization point includes a second scheduling operation that is performed to copy the output dataset of the workload skewing operation into one or more buffers and determine a second task size for one or more second operations following the second scheduling operation in the query pipeline based on a second quantity of time required to perform the one or more second operations on a second portion of the output dataset smaller than the first portion of the output dataset, wherein the second scheduling operation is further configured to schedule, based at least on the second task size, one or more corresponding tasks for performance by a corresponding quantity of threads once the one or more buffers are full;
• repeating, at each of multiple subsequent points of the query pipeline, the determining and the inserting to insert one or more additional reparallelization points in the query pipeline in response to determining that one or more additional operations is a workload skewing operation;
• and
• executing the query by at least performing the sequence of operations comprising the query pipeline.

A non-transitory computer readable medium storing instructions, which when executed by at least one data processor, result in operations comprising:

• determining whether an operation within a query pipeline is a workload skewing operation, wherein the query pipeline is an architecture in which a query is divided into a sequence of operations for execution of the query, wherein a size of an output dataset of the workload skewing operation is more than a threshold larger or more than a threshold smaller than an input dataset of the workload skewing operation, and wherein the query pipeline comprises one or more first operations following a first scheduling operation that is performed to determine a first task size for the one or more first operations based on a first quantity of time required to perform the one or more first operations on a first portion of the output dataset;
• in response to determining that an operation is a workload skewing operation, inserting, in the query pipeline, a reparallelization point subsequent to the workload skewing operation, wherein the reparallelization point includes a second scheduling operation that is performed to copy the output dataset of the workload skewing operation into one or more buffers and determine a second task size for one or more second operations following the second scheduling operation in the query pipeline based on a second quantity of time required to perform the one or more second operations on a second portion of the output dataset smaller than the first portion of the output dataset, wherein the second scheduling operation is further configured to schedule, based at least on the second task size, one or more corresponding tasks for performance by a corresponding quantity of threads once the one or more buffers are full;
• repeating, at each of multiple subsequent points of the query pipeline, the determining and the inserting to insert one or more additional reparallelization points in the query pipeline in response to determining that one or more additional operations is a workload skewing operation;
• and
• executing the query by at least performing the sequence of operations comprising the query pipeline.