Dynamic computation node grouping with cost based optimization for massively parallel processing

A computer-implemented method for accessing data in a massively parallel processing shared nothing relational database management system, the method comprising:

• storing a set of data in a first set of storages in a plurality of storage, the first set of storages assigned to a first set of compute nodes in a plurality of compute nodes;
• obtaining a plurality of logic plans that include the database operation on the set of data stored in the first set of storages to respond to a query that access the set of data;
• determining a cost of redistributing the set of data stored in the storage assigned to the first set of compute nodes to a second set of compute nodes for each logic plan in the plurality of logic plans;
• determining a cost reduction from inter-partition parallelism for each logic plan in the plurality of logic plans;
• selecting a logic plan from the plurality of logic plans based on the cost of redistributing the set of data and the cost reduction from inter-partition parallelism;
• repartitioning the set of data by hashing into a repartitioned set of data;
• reassigning the first set of storages to the second set of compute nodes in the plurality of compute nodes;
• distributing the repartitioned set of data to the second set of compute nodes;
• and
• performing a database operation on the repartitioned set of data by the second set of compute nodes using the selected logic plan to provide an answer to the query.

A non-transitory computer-readable medium storing computer instructions for accessing data in a massively parallel processing shared nothing relational database management system, that when executed by one or more processors, cause one or more processors to perform the steps of:

• store a set of data in a first set of storages in a plurality of storages, the first set of storages assigned to a first set of compute nodes in a plurality of compute nodes;
• obtain a plurality of logic plans that include the database operation on the set of data stored in the first set of storages to respond to a query that accesses the set of data;
• determine a cost of redistributing the set of data stored in the first set of storages to a second set of compute nodes for each logic plan in the plurality of logic plans;
• determine a cost reduction from inter-partition parallelism for each logic plan in the plurality of logic plans;
• select a logic plan from the plurality of logic plans based on the cost of redistributing the set of data and the cost reduction from inter-partition parallelism;
• repartition the set of data by hashing into a repartitioned set of data;
• reassign the first set of storages to the second set of compute nodes;
• distribute the repartitioned set of data to the second set of compute nodes;
• and
• perform a database operation on the repartitioned set of data by the second set of compute nodes to provide an answer to the query.