# US 2021/0081184 A1 · Method and apparatus for enabling autonomous acceleration of dataflow ai applications

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0081184 A1
- Document ID: US20210081184A1
- Status: Published application
- Kind: A1
- Application number: 16/570,822
- Filed: 2019-09-13
- Published: 2021-03-18
- Earliest claimed: 2019-09-13
- Assignees: Huawei Technologies Co., Ltd.
- Applicants: Huawei Technologies Co., Ltd.
- Inventors: Reza AZIMI; Kai-Ting Amy WANG; Yaoqing GAO; Ye TIAN; Xiang WANG; Cheng Xiang FENG
- Web page: https://patentgrep.com/patent/US20210081184A1

## CPC

- G06F 8/4441: {Reducing the execution time required by the program code}
- G06F 8/451: {Code distribution (considering CPU load at run-time G06F9/505; load rebalancing G06F9/5083)}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A method includes analyzing a dataflow graph representing data dependencies between operators of a dataflow application to identify a plurality of candidate groups of the operators. Based on characteristics of a given hardware accelerator and the operators of a given candidate group of the plurality of candidate groups, determining whether the operators of the given candidate group are to be combined. In response to determining that the operators of the given candidate group are to be combined, retrieving executable binary code segments corresponding to the operators of the given candidate group, generating a unit of binary code including the executable binary code segments and metadata representing an execution control flow among the executable binary code segments, and dispatching the unit of code to the given hardware accelerator for execution of the unit of code.

## Claims

### Claim 1

A method performed by a software compiler, the method comprising:
analyzing a dataflow graph representing data dependencies between operators of a dataflow application to identify a plurality of candidate groups of the operators;
based on characteristics of a given hardware accelerator and the operators of a given candidate group of the plurality of candidate groups, determining whether the operators of the given candidate group are to be combined; and
in response to determining that the operators of the given candidate group are to be combined:
retrieving executable binary code segments corresponding to the operators of the given candidate group;
generating a unit of binary code including the executable binary code segments and metadata representing an execution control flow among the executable binary code segments; and
dispatching the unit of code to the given hardware accelerator for execution of the unit of code.

### Claim 2 (depends on 1)

The method of claim 1, wherein determining whether the operators of the given candidate group are to be grouped together comprises determining whether the operators should be combined based on whether operators of the given candidate group are constrained to be executed on different hardware accelerators of a plurality of hardware accelerators.

### Claim 3 (depends on 1)

The method of claim 1, wherein determining whether the operators of the given candidate group are to be grouped together comprises determining whether the operators should be combined based on available resources of the given hardware accelerator.

### Claim 4 (depends on 1)

The method of claim 1, wherein determining whether the operators of the given candidate group are to be grouped together comprises determining whether the operators should be combined based on a number of the operators of the given candidate group.

### Claim 5 (depends on 1)

The method of claim 1, wherein determining whether the operators of the given candidate group are to be grouped together comprises determining whether the operators should be combined based on a computation runtime associated with executing all of the operators of the given candidate group.

### Claim 6 (depends on 1)

The method of claim 1, wherein generating the unit of binary code comprises generating metadata representing input parameters and output parameters for each operator of the given candidate group.

### Claim 7 (depends on 1)

The method of claim 1, wherein generating the unit of binary code comprises generating metadata representing a plurality of navigation table entries, wherein a given entry of the plurality of navigation entries comprises a field associated with a size of the given entry and a pointer to a given executable binary code segment of the plurality of executable binary code segments.

### Claim 8 (depends on 1)

The method of claim 1, wherein the unit of binary code comprises glue code.

### Claim 9

A computer system comprising:
a processor;
a hardware accelerator; and
a non-transient memory for storing instructions which when executed by the processor cause the processor to:
execute a unit of binary code including executable binary code segments and metadata representing an execution control flow among the executable binary code segments, the executable binary code segments corresponding to operators of a given candidate group, the operators of the given candidate group selected based on, analyzing a dataflow graph representing data dependencies between operators of a dataflow application to identify a plurality of candidate groups of the operators, and based on characteristics of the hardware accelerator and the operators of the given candidate group of the plurality of candidate groups, determining that the operators of the given candidate group are to be combined; and
dispatch the unit of code to the hardware accelerator for execution of the unit of code.

### Claim 10 (depends on 9)

The computer system of claim 9, wherein determining that the operators of the given candidate group are to be grouped together comprises determining that the operators are combined based on constraints of the hardware accelerator.

### Claim 11 (depends on 9)

The computer system of claim 9, wherein determining that the operators of the given candidate group are to be grouped together comprises determining that the operators should be combined based on available resources of the hardware accelerator.

### Claim 12 (depends on 9)

The computer system of claim 9, wherein determining that the operators of the given candidate group are to be grouped together comprises determining that the operators should be combined based on a number of the operators of the given candidate group.

### Claim 13 (depends on 9)

The computer system of claim 9, wherein determining that the operators of the given candidate group are to be grouped together comprises determining that the operators should be combined based on a computation runtime associated with executing all of the operators of the given candidate group.

### Claim 14 (depends on 9)

The computer system of claim 9, wherein generating the unit of binary code comprises generating metadata representing input parameters and output parameters for each operator of the given candidate group.

### Claim 15 (depends on 9)

The computer system of claim 9, wherein generating the unit of binary code comprises generating metadata representing a plurality of navigation table entries, wherein a given entry of the plurality of navigation entries comprises a field associated with a size of the given entry and a pointer to a given executable binary code segment of the plurality of executable binary code segments.

### Claim 16 (depends on 9)

The computer system of claim 9, wherein the unit of binary code comprises glue code.

### Claim 17

A non-transient memory having stored therein instructions, the instructions when executed by a processor causing the processor to:
execute a unit of binary code including executable binary code segments and metadata representing an execution control flow among the executable binary code segments, the executable binary code segments corresponding to operators of a given candidate group, the operators of the given candidate group selected based on, analyzing a dataflow graph representing data dependencies between operators of a dataflow application to identify a plurality of candidate groups of the operators, and based on characteristics of a hardware accelerator and the operators of the given candidate group of the plurality of candidate groups, determining that the operators of the given candidate group are to be combined; and
dispatch the unit of code to the hardware accelerator for execution of the unit of code.

### Claim 18 (depends on 17)

The non-transient memory of claim 17, wherein determining that the operators of the given candidate group are to be grouped together comprises determining that the operators are combined based on constraints of the hardware accelerator.

### Claim 19 (depends on 17)

The non-transient memory of claim 17, wherein determining that the operators of the given candidate group are to be grouped together comprises determining that the operators should be combined based on a computation runtime associated with executing all of the operators of the given candidate group.

### Claim 20 (depends on 17)

The non-transient memory of claim 17, wherein determining that the operators of the given candidate group are to be grouped together comprises determining that the operators should be combined based on a computation runtime associated with executing all of the operators of the given candidate group.
