System and method for scalable data processing operations

A method for processing a sequence of statements of a code block, the statements including data processing operations, the method comprising:

• maintaining a queue that includes a translatable portion comprising indications of data processing operations translatable to data queries and a non-translatable portion comprising indications of non-translatable data processing operations;
• determining that a first data processing operation of a first code block statement is not translatable to a database query;
• including an indication of the first data processing operation in the non-translatable portion of the queue;
• responsive to a determination that a second data processing operation of a second code block statement is undeferrable, compiling the translatable portion of the queue into a database query;
• causing the database query to be executed by a database engine to generate a query result;
• causing the first data processing operation to be executed by a data processing engine to generate a first data processing result;
• and
• transmitting, to an application configured to analyze the result dataset, a result dataset corresponding to the query result and the first data processing result.

A computer-readable storage medium encoded with program instructions that, when executed by a processor circuit, perform a method for processing a sequence of statements of a code block, the method comprising:

• maintaining a queue that includes a translatable portion comprising indications of data processing operations translatable to data queries and a non-translatable portion comprising indications of non-translatable data processing operations;
• determining that a first data processing operation of a first code block statement is not translatable to a database query;
• including an indication of the first data processing operation in the non-translatable portion of the queue;
• responsive to a determination that a second data processing operation of a second code block statement is undeferrable, compiling the translatable portion of the queue into a database query;
• causing the database query to be executed by a database engine to generate a query result;
• causing the first data processing operation to be executed by a data processing engine to generate a first data processing result;
• and
• transmitting, to an application configured to analyze the result dataset, a result dataset corresponding to the query result and the first data processing result.