Query execution pipelining with shared states for query operators

A method comprising:

• receiving, by a database execution engine, a query from a client device;
• identifying, by the database execution engine, a plurality of operators of a query plan for executing a query, the plurality of operators including a first operator and a second operator, the database execution engine including a first interface to one or more client devices, a second interface to one or more databases, and a third interface to at least one other database execution engine, the database execution engine including a query optimizer and a query execution engine separate from the one or more databases;
• generating, by the database execution engine and based on a first coded class definition, the first operator;
• generating, by the database execution engine and based on a second coded class definition, the second operator of a second query execution pipeline;
• generating, by the database execution engine and based on the first coded class definition and the second coded class definition, an indication that the first operator and the second operator share first data, the first data being intermediate result data generated during execution of the query plan and not available at a persistent layer of the one or more databases when the query is initially received;
• executing, by the database execution engine, the first operator to generate at least a portion of the first data;
• storing, by the database execution engine, the first data in a hash table rather than store the first data in a database table at the persistent layer of the one or more databases;
• and
• providing, by the database execution engine and based upon the indication, to the second operator of the second query execution pipeline the first data from the hash table, the first data serving as input data for execution of the second operator.

A non-transitory computer program product storing instructions which, when executed by at least one data processor, causes operations comprising:

• receiving, by a database execution engine, a query from a client device;
• identifying, by the database execution engine, a plurality of operators of a query plan for executing a query, the plurality of operators including a first operator and a second operator, the database execution engine including a first interface to one or more client devices, a second interface to one or more databases, and a third interface to at least one other database execution engine, the database execution engine including a query optimizer and a query execution engine separate from the one or more databases;
• generating, by the database execution engine and based on a first coded class definition, the first operator;
• generating, by the database execution engine and based on a second coded class definition, the second operator of a second query execution pipeline;
• generating, by the database execution engine and based on the first coded class definition and the second coded class definition, an indication that the first operator and the second operator share first data, the first data being intermediate result data generated during execution of the query plan and not available at a persistent layer of the one or more databases when the query is initially received;
• executing, by the database execution engine, the first operator to generate at least a portion of the first data;
• storing, by the database execution engine, the first data in a hash table rather than store the first data in a database table at the persistent layer of the one or more databases;
• and
• providing, by the database execution engine and based upon the indication, to the second operator of the second query execution pipeline the first data from the hash table, the first data serving as input data for execution of the second operator.