Method and system for providing a context-sensitive, non-intrusive data processing optimization framework

A system for performing, by a query processing engine of specific hardware, a data search in a database comprising:

• a memory storing instructions;
• and
• at least one processor in communication with the memory, the at least one processor configured, upon execution of the instructions, to perform the following steps:
• obtaining intermediate representation code of at least one operator, the intermediate representation code configured in a high-level programming language and compiled by a front-end compiler;
• optimizing the intermediate representation code, based on information of the runtime of performing the data search;
• compiling the optimized intermediate representation code to generate at least one optimized and compiled operator, and
• performing the data search by the query processing engine using the at least one optimized and compiled operator,
• wherein the information of the runtime of performing the data search comprises statistical information about the data search or the data related thereto;
• and
• wherein the optimizing the intermediate representation code comprises:
• specializing, based on the information of the runtime of performing the data search, the intermediate representation code for adapting to the data search and the hardware of the query processing engine,
• selecting a level of parallelism and an operator fusion mechanism using the statistical information, and
• optimizing the specialized intermediate representation code, based on the information of the runtime of performing the data search, to generate at least one optimized and compiled operator.

A non-transitory computer-readable media storing comprising computer instructions that configure at least one processor, upon execution of the instructions, to perform the following steps:

• obtain intermediate representation code of at least one operator, the intermediate representation code configured in a high-level programming language and compiled by a front-end compiler;
• optimized the intermediate representation code, based on information of the runtime of performing the data search;
• compile the optimized intermediate representation code to generate at least one optimized and compiled operator;
• and
• perform the data search by the query pr si engine using the at least one optimized and compiled operator;
• wherein the information of the runtime of performing the data search comprises statistical information about the data search or the data related thereto;
• and
• wherein the optimizing specializing, based on the information of the runtime of performing the data search the intermediate representation code for adapting to the data search and the hardware of the query processing engine,
• selecting a level of parallelism and an operator fusion mechanism using the statistical information, and
• optimizing the specialized intermediate representation code, based on the information of the runtime of performing the data search, to generate at least one optimized and compiled operator.