Database distribution system and methods for scale-out applications

A method for query optimization in a scale-out system with a single query processing machine and a distributed storage engine to store data, the method comprising:

• receiving a query rewritten for an internal schema;
• optimizing a query execution plan for the query;
• executing the query execution plan and returning result of the execution to an application;
• conforming to an access limitation imposed by the storage engine where no relation R can be visited before at least one other relation with one or more pointers to R is accessed;
• and
• elevating each relation R when every relation with filtering power over R is accessed before R,
• wherein the optimization includes:
• determining a first executable plan with a minimum number of requests to the distributed storage engine if an effective parallelism available to the query engine is below a predetermined threshold;
• and
• determining a second executable plan with a smallest depth, regardless of the total number of requests to be issued, if the effective parallelism exceeds the predetermined threshold, and
• wherein the query execution plan is configured to support intra-operator parallelism, inter-operator parallelism, and pipeline parallelism.