# US 12,761,965 B2 · Code objects offload based on accelerator, accelerator cache and data transfer metrics

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,761,965 B2
- Document ID: US12761965B2
- Status: Granted
- Kind: B2
- Application number: 18/030,057
- Filed: 2021-04-23
- Published: 2023-11-16
- Granted: 2026-10-13
- Earliest claimed: 2020-12-08
- Assignees: Intel Corporation
- Applicants: Intel Corporation
- Inventors: Kermin E. ChoFleming, Jr.; Egor A. Kazachkov; Daya Shanker Khudia; Zakhar A. Matveev; Sergey U. Kokljuev; Fabrizio Petrini; Dmitry S. Petrov; Swapna Raj
- Web page: https://patentgrep.com/patent/US12761965B2

## CPC

- G06F 9/5044: {considering hardware capabilities}
- G06F 11/302: {where the computing system component is a software system}
- G06F 11/3409: {for performance assessment}
- G06F 2201/865: Monitoring of software
- G06F 2209/509: Offload
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

An offload analyzer analyzes a program for porting to a heterogeneous computing system by identifying code objects for offloading to an accelerator. Runtime metrics generated by executing the program on a host processor unit are provided to an accelerator model that models the performance of the accelerator and generates estimated accelerator metrics for the program. A code object offload selector selects code objects for offloading based on whether estimated accelerated times of the code objects, which comprise estimated accelerator times and offload overhead times, are better than their host processor unit execution times. The code object offload selector selects additional code objects for offloading using a dynamic-programming-like performance estimation approach that performs a bottom-up traversal of a call tree. A heterogeneous version of the program can be generated for execution on the heterogeneous computing system.

## Claims

### Claim 1

A method, comprising:
generating runtime metrics for a program comprising a plurality of code objects, the runtime metrics indicating performance of the program executing on a host processor unit, wherein generating the runtime metrics comprises executing the program on the host processor unit in a non-offloaded mode in which the plurality of code objects are executed on the host processor unit and not on an accelerator;
generating, utilizing an accelerator cache model, modeled accelerator cache metrics based on the runtime metrics;
generating, utilizing a data transfer model, modeled data transfer metrics based on the runtime metrics;
generating, utilizing an accelerator model, estimated accelerator metrics based on the runtime metrics and the modeled accelerator cache metrics;
selecting one or more code objects of the plurality of code objects for offloading to an accelerator based on the estimated accelerator metrics, the modeled data transfer metrics, and the runtime metrics; and
generating a heterogeneous program comprising the one or more code objects selected for offloading and one or more code objects of the plurality of code objects not selected for offloading that, when executed on a heterogeneous computing system comprising a target host processor unit and a target accelerator of the accelerator, executes the one or more code objects not selected for offloading on the target host processor unit and offloads the one or more code objects selected for offloading to the target accelerator.

### Claim 2 (depends on 1)

The method of claim 1, wherein the generating the estimated accelerator metrics comprises, for individual of the plurality of code objects, determining an estimated accelerated time, an estimated accelerator execution time, and an estimated offload overhead time, wherein the estimated accelerated time is the estimated accelerator execution time plus the estimated offload overhead time.

### Claim 3 (depends on 2)

The method of claim 2, wherein the determining the estimated accelerator execution time for individual of the plurality of code objects comprises:
determining an estimated compute-bound accelerator execution time for the individual code object based on one or more of the runtime metrics;
determining one or more estimated memory-bound accelerator execution times for the individual code object based on one or more of the modeled accelerator cache metrics, individual of the one or more estimated memory-bound accelerator execution times corresponding to a memory hierarchy level of a memory hierarchy available to the accelerator; and
selecting a maximum of the estimated compute-bound accelerator execution time and the one or more estimated memory-bound accelerator execution times as the estimated accelerator execution time for the individual code object.

### Claim 4 (depends on 3)

The method of claim 3, wherein the determining the estimated compute-bound accelerator execution time comprises, for the individual code objects that are loops:
determining a plurality of estimated compute-bound loop accelerator execution times for the individual code object, individual of the estimated compute-bound loop accelerator execution times based on a loop unroll factor from a plurality a of different loop unroll factors; and
setting the estimated compute-bound accelerator execution time for the individual code object to a minimum of the estimated compute-bound loop accelerator execution times.

### Claim 5 (depends on 3)

The method of claim 3, wherein the determining the estimated offload overhead time for the individual code object is based on one or more of the modeled data transfer metrics associated with the individual code object.

### Claim 6 (depends on 3)

The method of claim 3, wherein the runtime metrics comprise a host processor unit execution time for individual of the plurality of code objects, the selecting the one or more code objects for offloading comprising selecting as the code objects for offloading those code objects for which the estimated accelerated time is less than the host processor unit execution time.

### Claim 7 (depends on 3)

The method of claim 3, wherein the runtime metrics comprise a host processor unit execution time for individual of the plurality of code objects, the selecting the code objects for offloading comprising performing a bottom-up traversal of a call tree of the program, individual nodes of the call tree corresponding to one of the code objects, for individual nodes in the call tree reached during the bottom-up traversal:
(i) determining a first estimated execution time, the first estimated execution time a sum of a total estimated offload overhead time for the code objects associated with the individual node and children nodes of the individual node being considered as offloaded together, and a total estimated accelerator execution time for the code objects associated with the individual node and the children nodes of the individual node being considered as offloaded together;
(ii) summing the host processor unit execution times for the code objects associated with the individual node and the children nodes of the individual node to determine a second estimated execution time;
(iii) determining a third estimated execution time for the code objects associated with the individual node and children nodes of the individual node if the code object associated with the individual node were to be executed on the host processor unit and the code objects associated with the children nodes of the individual node were executed on either the host processor unit or the accelerator based on which code objects associated with the children nodes were selected for offloading prior to performing (i), (ii) and (iii) for the individual node, the determining the third estimated execution time comprising summing a total estimated offload overhead time for the code objects associated with the children nodes of the individual node selected for offloading prior to performing (i), (ii), and (iii) being considered as offloaded together, a host processor unit execution time for the code object associated with the individual node, and a total estimated execution time for the children nodes of the individual node determined prior to performing (i), (ii), and (iii) for the individual node;
(iv) if the first estimated execution time is a minimum of the first estimated execution time, the second estimated execution time, and the third estimated execution time, selecting the first estimated execution time as an estimated execution time of the individual node and selecting the code objects associated with the individual node and the children nodes of the individual node for offloading;
(v) if the second estimated execution time is a minimum of the first estimated execution time, the second estimated execution time, and the third estimated execution time, selecting the second estimated execution time as the estimated execution time of the individual node and unselecting the code objects associated with the individual node and the children nodes of the individual node for offloading; and
(vi) if the third estimated execution time is a minimum of the first estimated execution time, the second estimated execution time, and the third estimated execution time, selecting the third estimated execution time as the estimated execution time of the individual node.

### Claim 8 (depends on 1)

The method of claim 1, further comprising causing the heterogeneous program to be executed on the heterogeneous computing system.

### Claim 9 (depends on 1)

The method of claim 1, wherein the modeled data transfer metrics comprise an estimated data transfer overhead between the host processor unit and an accelerator for at least one of the code objects.

### Claim 10

A computing system comprising:
one or more processors; and
one or more non-transitory computer-readable storage media having instructions stored thereon that, when executed, cause the one or more processors to:
generate runtime metrics for a program comprising a plurality of code objects, the runtime metrics indicating performance of the program executing on a host processor unit, wherein generating the runtime metrics comprises executing the program on the host processor unit in a non-offloaded mode in which the plurality of code objects are executed on the host processor unit and not on an accelerator;
generate, utilizing an accelerator cache model, modeled accelerator cache metrics based on the runtime metrics;
generate, utilizing a data transfer model, modeled data transfer metrics based on the runtime metrics;
generate, utilizing an accelerator model, estimated accelerator metrics based on the runtime metrics and the modeled accelerator cache metrics;
select one or more code objects of the plurality of code objects for offloading to an accelerator based on the estimated accelerator metrics, the modeled data transfer metrics, and the runtime metrics; and
generate a heterogeneous program comprising the one or more code objects selected for offloading and one or more code objects of the plurality of code objects not selected for offloading that, when executed on a heterogeneous computing system comprising a target host processor unit and a target accelerator of the accelerator, executes the one or more code objects not selected for offloading on the target host processor unit and offloads the one or more code objects selected for offloading to the target accelerator.

### Claim 11 (depends on 10)

The computing system of claim 10, wherein to generate the estimated accelerator metrics comprises, for individual of the plurality of code objects, to determine an estimated accelerated time, an estimated accelerator execution time, and an estimated offload overhead time, wherein the estimated accelerated time is the estimated accelerator execution time plus the estimated offload overhead time.

### Claim 12

One or more non-transitory computer-readable storage media storing computer-executable instructions for causing a computing system to:
generate runtime metrics for a program comprising a plurality of code objects, the runtime metrics indicating performance of the program executing on a host processor unit, wherein generating the runtime metrics comprises executing the program on the host processor unit in a non-offloaded mode in which the plurality of code objects are executed on the host processor unit and not on an accelerator;
generate, utilizing an accelerator cache model, modeled accelerator cache metrics based on the runtime metrics;
generate, utilizing a data transfer model, modeled data transfer metrics based on the runtime metrics;
generate, utilizing an accelerator model, estimated accelerator metrics based on the runtime metrics and the modeled accelerator cache metrics;
select one or more code objects of the plurality of code objects for offloading to an accelerator based on the estimated accelerator metrics, the modeled data transfer metrics, and the runtime metrics; and
generate a heterogeneous program comprising the one or more code objects selected for offloading and one or more code objects of the plurality of code objects not selected for offloading that, when executed on a heterogeneous computing system comprising a target host processor unit and a target accelerator of the accelerator, executes the one or more code objects not selected for offloading on the target host processor unit and offloads the one or more code objects selected for offloading to the target accelerator.

### Claim 13 (depends on 12)

The one or more non-transitory computer-readable storage media of claim 12, to generate the estimated accelerator metrics comprising, for individual of the plurality of code objects, to determine an estimated accelerated time, an estimated accelerator execution time, and an estimated offload overhead time, wherein the estimated accelerated time is the estimated accelerator execution time plus the estimated offload overhead time.

### Claim 14 (depends on 13)

The one or more non-transitory computer-readable storage media of claim 13, to determine the estimated accelerator execution time for individual of the plurality of code objects comprising to:
determine an estimated compute-bound accelerator execution time for the individual code object based on one or more of the runtime metrics;
determine one or more estimated memory-bound accelerator execution times for the individual code object based on one or more of the modeled accelerator cache metrics, individual of the estimated memory-bound accelerator execution times corresponding to a memory hierarchy level of a memory hierarchy available to the accelerator; and
to select a maximum of the estimated compute-bound accelerator execution time and the estimated memory-bound accelerator execution times as the estimated accelerator execution time for the individual code object.

### Claim 15 (depends on 14)

The one or more non-transitory computer-readable storage media of claim 14, to determine the estimated compute-bound accelerator execution time comprising, for the individual code objects that are loops:
to determine a plurality of estimated compute-bound loop accelerator execution times for the individual code object, individual of the estimated compute-bound loop accelerator execution times based on a loop unroll factor from a plurality a of different loop unroll factors; and
to set the estimated compute-bound accelerator execution time for the individual code object to a minimum of the estimated compute-bound loop accelerator execution times.

### Claim 16 (depends on 13)

The one or more non-transitory computer-readable storage media of claim 13, wherein to determine the estimated offload overhead time for the individual code object is based on one or more of the modeled data transfer metrics associated with the individual code object.

### Claim 17 (depends on 13)

The one or more non-transitory computer-readable storage media of claim 13, wherein the runtime metrics comprise a host processor unit execution time for individual of the plurality of code objects, to select the one or more code objects for offloading comprising to select as the one or more code objects for offloading those code objects for which the estimated accelerated time is less than the host processor unit execution time.

### Claim 18 (depends on 13)

The one or more non-transitory computer-readable storage media of claim 13, wherein the accelerator model is a first accelerator model that models behavior of a first accelerator, to generate the estimated accelerator metrics utilizing the first accelerator model and a second accelerator model that models behavior of a second accelerator to generate the estimated accelerator metrics based on the modeled accelerator cache metric, and the runtime metrics, wherein the estimated accelerator metrics comprise, for individual of the plurality of code objects, an estimated accelerated time for the first accelerator and an estimated accelerator time for the second accelerator.

### Claim 19 (depends on 18)

The one or more non-transitory computer-readable storage media of claim 18, wherein the runtime metrics comprise a host processor unit execution time for individual of the plurality of code objects, to select the one or more code objects for offloading comprising to:
select as code objects for offloading to the first accelerator those code objects for which the estimated accelerated time for the first accelerator is less than the host processor unit execution time; and
select as code objects for offloading to the second accelerator those code objects for which the estimated accelerated time for the second accelerator is less than the host processor unit execution time.

### Claim 20 (depends on 12)

The one or more non-transitory computer-readable storage media of claim 12, wherein the modeled data transfer metrics comprise an estimated data transfer overhead between the host processor unit and an accelerator for at least one of the code objects.