Method, apparatus, and readable storage medium for leveraging data formats based on operators to accelerate data processing

An apparatus applied to a database system, comprising:

• one or more processors;
• and
• memory storing computer-executable instructions that, when executed by the one or more processors, cause the apparatus to:
• receive, by a query parser of the database system, a structured query language (SQL) data processing request that indicates processing on data in a database, the processing including first data processing and second data processing, the first data processing being different from the second data processing;
• generate, by a query optimizer of the database system, a physical plan in response to the SQL data processing request, the physical plan comprising a first physical operator, or comprising the first physical operator and a second physical operator;
• determine a first data format based on a first reading order in which a plurality of first data to be processed by a first phase of the first physical operator is read during the first data processing, wherein the first data format is selected from a row-based format, a column-based format, or a hybrid format;
• convert the plurality of first data into the first data format;
• execute the first data processing on the converted plurality of first data via the first phase of the first physical operator to generate a result of the first data processing;
• determine a second data format based on a second reading order in which a plurality of second data to be processed by a second phase of the first physical operator or the second physical operator is read during the second data processing, wherein the second data format is selected from the row-based format, the column-based format, or the hybrid format, and the second data format is different from the first data format, the plurality of second data includes the result of the first data processing;
• convert the plurality of second data into the second data format;
• and
• execute the second data processing on the converted plurality of second data via the second phase of the first physical operator or via the second physical operator,
• wherein the first data processing has a higher execution efficiency for the first data format than for the second data format, and the second data processing has a higher execution efficiency for the second data format than for the first data format, and
• wherein converting the plurality of first data and converting the plurality of second data occur in real-time during query execution without pre-storing data in multiple formats.

A non-transitory computer-readable storage medium storing instructions, which when executed by one or more processors, cause a device to:

• receive, by a query parser of a database system, a structured query language (SQL) data processing request that indicates processing on data in a database, the processing including first data processing and second data processing, the first data processing being different from the second data processing;
• generate, by a query optimizer of the database system, a physical plan in response to the SQL data processing request, the physical plan comprising a first physical operator, or comprising the first physical operator and a second physical operator;
• determine a first data format based on a first reading order in which a plurality of first data to be processed by a first phase of the first physical operator is read during the first data processing, wherein the first data format is selected from a row-based format, a column-based format, or a hybrid format;
• convert the plurality of first data into the first data format;
• execute the first data processing on the converted plurality of first data via the first phase of the first physical operator to generate a result of the first data processing;
• determine a second data format based on a second reading order in which a plurality of second data to be processed by a second phase of the first physical operator or the second physical operator is read during the second data processing, wherein the second data format is selected from the row-based format, the column-based format, or the hybrid format, and the second data format is different from the first data format, the plurality of second data includes the result of the first data processing;
• convert the plurality of second data into the second data format;
• and
• execute the second data processing on the converted plurality of second data via the second phase of the first physical operator or via the second physical operator,
• wherein the first data processing has a higher execution efficiency for the first data format than for the second data format, and the second data processing has a higher execution efficiency for the second data format than for the first data format, and
• wherein converting the plurality of first data and converting the plurality of second data occur in real-time during query execution without pre-storing data in multiple formats.