Dynamic workload management optimizations using real-time execution feedback

A system comprising:

• a distributed query processor;
• a query optimizer;
• a cluster of nodes, the cluster of nodes comprising a first node and a second node;
• and
• a memory comprising computer-readable media that, when executed by the distributed query processor, cause the distributed query processor to perform the following operations:
• receiving a query;
• identifying a plurality of tasks to execute the query, the plurality of tasks comprising a first task and a second task that depends from the first task;
• receiving, from the query optimizer, a first estimation of a first amount of resources to be used to execute the first task;
• receiving, from the query optimizer, a second estimation of a second amount of resources to be used to execute the second task;
• based at least on the first estimation and the second estimation, assigning the first task to the first node and the second task to the second node;
• and
• prior to the second task on the second node being executed:
• receiving, from the first node, an actual amount of resources used to execute the first task;
• and
• based at least on the actual amount of the resources used to execute the first task, adjusting the second estimation of the second amount of resources to be used to execute the second task.

One or more computer-readable media comprising computer-executable instruction that, when executed by a processor, cause the processor to perform the following operations:

• receiving a query;
• identifying a plurality of tasks to execute the query, the plurality of tasks comprising a first task and a second task that depends from the first task;
• providing a first estimation of a first amount of resources to be used to execute the first task;
• providing a second estimation of a second amount of resources to be used to execute the second task;
• based at least on the first estimation and the second estimation, assigning the first task to a first node and the second task to a second node;
• and
• prior to the second task on the second node being executed:
• receiving, from the first node, an actual amount of resources used to execute the first task;
• and
• based at least on the actual amount of the resources used to execute the first task, adjusting the second estimation of the second amount of resources to be used to execute the second task.