Dynamic usage balance of central processing units and accelerators

An article of manufacture comprising a computer-readable storage medium for storing computer readable program code which, when executed, causes a computer to:

• collect task information and accelerator information, wherein the accelerator information is associated with one or more accelerators;
• in response to receipt of a task request based on the collected task information and the collected accelerator information for a plurality of strategies for executing the workload, wherein at least two of the plurality of strategies are associated with a distribution of the workload between a central processing unit (CPU) and the one or more accelerators;
• select a strategy from the plurality of strategies having a maximum utility value;
• execute the task request according to the selected strategy;
• and
• update the task information and the accelerator information,
• wherein the task information comprises information selected from the group consisting of:
• speedup and memory allocation requirements, and wherein the accelerator information comprises information selected from the group consisting of:
• queue length, computational utilization, memory utilization, and socket affinity;
• and
• wherein said executing the task request according to the selected strategy comprises performing offloading the workload from the thread to an accelerator selected based on one or more considerations.

A system comprising:

• one or more accelerators operatively coupled to a central processing unit (CPU), wherein the CPU is configured to:
• collect task information and accelerator information, wherein the accelerator information is associated with the one or more accelerators;
• receive a task request associated with a workload;
• calculate a utility value based on the collected task information and the collected accelerator information for a plurality of strategies for executing the workload, wherein at least two of the plurality of strategies are associated with a distribution of the workload between a central processing unit (CPU) and the one or more accelerators;
• select a strategy from the plurality of strategies having a maximum utility value;
• execute the task is executed according to the selected strategy;
• and
• update the task information and the accelerator information,
• wherein the task information comprises information selected from the group consisting of:
• speedup and memory allocation requirements, and wherein the accelerator information comprises information selected from the group consisting of:
• queue length, computational utilization, memory utilization, and socket affinity;
• and
• wherein said executing the task request according to the selected strategy comprises performing offloading the workload from the thread to an accelerator selected based on one or more considerations.