Data block processing

A computing system comprising a computer processor coupled to a computer-readable memory unit, the memory unit comprising instructions that when executed by the computer processor implements a method comprising:

• receiving a request to execute a data processing task on a data object to generate a final result, the data object comprises a plurality of data blocks, and the data processing task comprises performing a computation on the data object to generate the final result;
• obtaining a first performance indicator indicative of a cost to use a first processing module to perform the computation of the data processing task on a particular data block loaded into the first processing module, wherein using the first processing module to perform the computation of the data processing task on the particular block excludes without transmitting the particular data block from the first processing module to another processing module based on the particular data block being loaded into the first processing module, the first cost excludes a processing cost of the transmission of the particular data block from the first processing module to another processing module, wherein the particular data block is among the plurality of data blocks, and the computation of the data processing task on the particular data block results in generation of a temporary result, the temporary result is a portion of the final result;
• obtaining a second performance indicator indicative of a cost to use a second processing module to perform the computation of the data processing task on the particular data block loaded into the first processing module to generate the temporary result, wherein the first and second processing modules being logical instances launched in a distributed processing system for processing the plurality of data blocks, wherein using the second processing module to perform the computation on the particular data block includes a transmission of the particular data block from the first processing module to the second processing module, and the second cost includes a processing cost of the transmission of the particular data block from the first processing module to the second processing module;
• selecting one from the first and second processing modules for performing the computation of the data processing task on the particular data block loaded into the first processing module to generate the temporary result, wherein the selecting is based on a relationship between the first and second performance indicators;
• controlling the selected processing module to perform the computation of the data processing task on the particular data block loaded into the first processing module to generate the temporary result;
• and
• aggregating the temporary result to other temporary results generated by other processing modules launched in the distributed processing system to generate the final result.

A computer program product being tangibly stored on a non-transitory computer readable storage medium and comprising machine-executable instructions, the instructions, when executed on an electronic device, causing the electronic device to:

• receive a request to execute a data processing task on a data object to generate a final result, the data object comprises a plurality of data blocks, and the data processing task comprises performing a computation on the data object to generate the final result;
• obtain a first performance indicator indicative of a cost to use a first processing module to perform the computation of the data processing task on a particular data block loaded into the first processing module, wherein using the first processing module to perform the computation of the data processing task on the particular block excludes transmitting the particular data block from the first processing module to another processing module based on the particular data block being loaded into the first processing module, the first cost excludes a processing cost of the transmission of the particular data block from the first processing module to another processing module, wherein the particular data block is among the plurality of data blocks, and the computation of the data processing task on the particular data block results in generation of a temporary result, the temporary result is a portion of the final result, wherein using the first processing module to perform the computation on the particular data block does not includes a transmission of the particular data block from the first processing module to another processing module;
• obtain a second performance indicator indicative of a cost to use a second processing module to perform the computation of the data processing task on the particular data block loaded into the first processing module to generate the temporary result, wherein the first and second processing modules being logical instances launched in a distributed processing system for processing the plurality of data blocks, wherein using the second processing module to perform the computation on the particular data block requires a transmission of the particular data block from the first processing module to the second processing module, and the second cost includes a processing cost of the transmission of the particular data block from the first processing module to the second processing module;
• select one from the first and second processing modules for performing the computation of the data processing task on the particular data block loaded into the first processing module to generate the temporary result, wherein the selection is based on a relationship between the first and second performance indicators;
• control the selected processing module to perform the computation of the data processing task on the particular data block loaded into the first processing module to generate the temporary result;
• and
• aggregate the temporary result to other temporary results generated by other processing modules launched in the distributed processing system to generate the final result.