Heterogeneous computing system and method including analyzing expected costs of compute kernels

A computer-readable non-transitory storage medium comprising instructions which, when executed by a heterogeneous computing system comprising a first processing unit and a second processing unit cause the computing system to carry out a method comprising:

• initializing a first processing unit of a heterogeneous computing system with a first compute kernel and a second processing unit of the heterogeneous computing system with a second compute kernel, wherein each of the first compute kernel and the second compute kernel is configured to perform a numerical operation derived from a program segment configured to receive a first data structure storing multiple elements of a common data type, the program segment comprising a function meta information comprising data related to a size of an output of the numerical operation, a structure of the output, and/or an effort for generating the output;
• using the function meta information and a data meta information of a runtime instance of the first data structure to numerically calculate first expected costs of executing the first compute kernel on the first processing unit to perform the numerical operation with the runtime instance and to numerically calculate second expected costs of executing the second compute kernel on the second processing unit to perform the numerical operation with the runtime instance, wherein the data meta information comprises a runtime size information of the runtime instance and a runtime location information of the runtime instance;
• and
• one of:
• executing the first compute kernel on the first processing unit to perform the numerical operation on the runtime instance if the first expected costs are lower than or equal to the second expected costs;
• and
• executing the second compute kernel on the second processing unit to perform the numerical operation on the runtime instance if the first expected costs are higher than the second expected costs.

A heterogeneous computing system comprising:

• a memory storing a runtime segment of a program segment configured to receive a first data structure capable to store multiple elements of a common data type, the program segment comprising a function meta information comprising data related to a size of an output of a numerical operation of the program segment, a structure of the output, and/or a numerical effort for generating the output;
• and
• at least two different processing units,
• wherein each of the at least two different processing units has access to and/or forms a part of the memory storing a respective compute kernel of the runtime segment, each compute kernel implementing the numerical operation,
• wherein the runtime segment comprises executable code for determining a data meta information of a runtime instance of the first data structure, the data meta information comprising a runtime size information of the runtime instance and a runtime location information of the runtime instance,
• wherein the runtime segment comprises executable code using the function meta information and the data meta information to calculate for each of the at least two different processing units respective expected costs of executing the respective compute kernels to perform the numerical operation with the runtime instance, and
• wherein the runtime segment comprises executable code for selecting one of the at least two different processing units for executing the respective compute kernel, the calculated expected costs of the selected one of the at least two processing units corresponding to a lowest value of the determined expected costs.