Processing of data chunks using a database calculation engine

A system comprising:

• at least one data processor;
• and,
• a memory coupled to the at least one data processor, the memory storing instructions, which, when executed, cause the at least one data processor to perform operations comprising:
• receiving, at the one or more data processors, a request to perform a primary query on a table stored in a database;
• determining, in response to the request, that the table exceeds a size limit of a main memory of a database management system associated with the database;
• and
• in response to the determination that the table exceeds the size limit of the main memory of the database management system,
• generating, by the one or more data processors, a first table query and a second table query, the first table query defining, based at least on a first starting row identifier and a first number of rows, a first data chunk from the table of the database, and the second table query defining, based at least on a second starting row identifier and a second number of rows, a second data chunk from the table of the database,
• executing the first table query to at least load, into the main memory, the first data chunk,
• performing the primary query on the first data chunk in the main memory, the primary query being performed on the first data chunk prior to executing the second table query to at least load, into the main memory, the second data chunk and performing the primary query on the second data chunk in the main memory, the loading of the second data chunk into the main memory overwriting the first data chunk in the main memory, and
• in response to the table including additional data in excess of the first data chunk and the second data chunk, generating and executing a third table query to at least load, into the main memory, a third data chunk including the additional data, and executing the primary query on the third data chunk in the main memory.

A non-transitory computer-readable medium storing instructions, which when executed by at least one data processor, result in operations comprising:

• receiving, at the one or more data processors, a request to perform a primary query on a table stored in a database;
• determining, in response to the request, that the table exceeds a size limit of a main memory of a database management system associated with the database;
• and
• in response to the determination that the table exceeds the size limit of the main memory of the database management system,
• generating, by the one or more data processors, a first table query and a second table query, the first table query defining, based at least on a first starting row identifier and a first number of rows, a first data chunk from the table of the database, and the second table query defining, based at least on a second starting row identifier and a second number of rows, a second data chunk from the table of the database,
• executing the first table query to at least load, into the main memory, the first data chunk,
• performing the primary query on the first data chunk in the main memory, the primary query being performed on the first data chunk prior to executing the second table query to at least load, into the main memory, the second data chunk and performing the primary query on the second data chunk in the main memory, the loading of the second data chunk into the main memory overwriting the first data chunk in the main memory, and
• in response to the table including additional data in excess of the first data chunk and the second data chunk, generating and executing a third table query to at least load, into the main memory, a third data chunk including the additional data, and executing the primary query on the third data chunk in the main memory.