Code objects offload based on accelerator, accelerator cache and data transfer metrics

A computing system comprising:

• one or more processors;
• and
• one or more non-transitory computer-readable storage media having instructions stored thereon that, when executed, cause the one or more processors to:
• generate runtime metrics for a program comprising a plurality of code objects, the runtime metrics indicating performance of the program executing on a host processor unit, wherein generating the runtime metrics comprises executing the program on the host processor unit in a non-offloaded mode in which the plurality of code objects are executed on the host processor unit and not on an accelerator;
• generate, utilizing an accelerator cache model, modeled accelerator cache metrics based on the runtime metrics;
• generate, utilizing a data transfer model, modeled data transfer metrics based on the runtime metrics;
• generate, utilizing an accelerator model, estimated accelerator metrics based on the runtime metrics and the modeled accelerator cache metrics;
• select one or more code objects of the plurality of code objects for offloading to an accelerator based on the estimated accelerator metrics, the modeled data transfer metrics, and the runtime metrics;
• and
• generate a heterogeneous program comprising the one or more code objects selected for offloading and one or more code objects of the plurality of code objects not selected for offloading that, when executed on a heterogeneous computing system comprising a target host processor unit and a target accelerator of the accelerator, executes the one or more code objects not selected for offloading on the target host processor unit and offloads the one or more code objects selected for offloading to the target accelerator.

One or more non-transitory computer-readable storage media storing computer-executable instructions for causing a computing system to:

• generate runtime metrics for a program comprising a plurality of code objects, the runtime metrics indicating performance of the program executing on a host processor unit, wherein generating the runtime metrics comprises executing the program on the host processor unit in a non-offloaded mode in which the plurality of code objects are executed on the host processor unit and not on an accelerator;
• generate, utilizing an accelerator cache model, modeled accelerator cache metrics based on the runtime metrics;
• generate, utilizing a data transfer model, modeled data transfer metrics based on the runtime metrics;
• generate, utilizing an accelerator model, estimated accelerator metrics based on the runtime metrics and the modeled accelerator cache metrics;
• select one or more code objects of the plurality of code objects for offloading to an accelerator based on the estimated accelerator metrics, the modeled data transfer metrics, and the runtime metrics;
• and
• generate a heterogeneous program comprising the one or more code objects selected for offloading and one or more code objects of the plurality of code objects not selected for offloading that, when executed on a heterogeneous computing system comprising a target host processor unit and a target accelerator of the accelerator, executes the one or more code objects not selected for offloading on the target host processor unit and offloads the one or more code objects selected for offloading to the target accelerator.