# US 2021/0081691 A1 · Efficient Execution of Operation Unit Graphs on Reconfigurable Architectures Based on User Specification

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0081691 A1
- Document ID: US20210081691A1
- Status: Published application
- Kind: A1
- Application number: 16/572,516
- Filed: 2019-09-16
- Published: 2021-03-18
- Earliest claimed: 2019-09-16
- Assignees: SambaNova Systems, Inc.
- Applicants: SambaNova Systems, Inc.
- Inventors: Zhuo CHEN; Sumti JAIRATH
- Web page: https://patentgrep.com/patent/US20210081691A1

## CPC

- G06F 15/7867: {with reconfigurable architecture}
- G06F 18/25: Fusion techniques
- G06N 3/048: Activation functions
- G06V 10/95: {structured as a network, e.g. client-server architectures}
- G06V 10/955: {using specific electronic processors}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

The technology disclosed relates to efficiently executing an operation unit graph on a reconfigurable data processor with a target architecture. In particular, it relates to reducing a number of physical compute units and/or physical memory units of the reconfigurable data processor required to execute the operation unit graph by receiving, from a user, architectural hints that are specific to the target architecture of the reconfigurable data processor, scanning the operation unit graph to detect instances of patterns of operation units specified by the architectural hints, and fusing operation units in the operation unit graph into a consolidated operation units block, thereby producing a fused operation unit graph.

## Claims

### Claim 1

A computer-implemented method of efficiently executing an operation unit graph on a reconfigurable data processor with a target architecture, the method including:
reducing a number of physical compute units and/or physical memory units of the reconfigurable data processor required to execute the operation unit graph by
receiving, from a user, architectural hints that are specific to the target architecture of the reconfigurable data processor,
wherein the architectural hints
call for fusing first operation units when executing patterns of the first operation units on the physical compute units and/or physical memory units of the reconfigurable data processor,
specify the first operation units in a pattern as first nodes,
specify first dataflows among the first operation units in the pattern as first edges, and
direct fusion among the first operation units in the pattern;
scanning the operation unit graph to detect instances of the patterns of the first operation units specified by the architectural hints, including
matching second nodes and second edges in the operation unit graph with the first nodes and the first edges in the architectural hints, and detecting pattern matches;
fusing operation units of the second nodes and the second edges in the operation unit graph into a consolidated operation units block, thereby producing a fused operation unit graph;
allocating the physical compute units and/or physical memory units of the reconfigurable data processor to the fused operation unit graph; and
executing the fused operation unit graph on the reconfigurable data processor based on the allocation.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein the architectural hints specify a first output operation unit in the pattern as a first output node.

### Claim 3 (depends on 2)

The computer-implemented method of claim 2, further including:
detecting the pattern matches by
matching the first output node specified by the architectural hints with a second output node in the operation unit graph, and
beginning with the second output node in the operation unit graph,
traversing the operation unit graph to determine that the second nodes and the second edges in the operation unit graph match the first nodes and the first edges in the architectural hints.

### Claim 4 (depends on 3)

The computer-implemented method of claim 3, wherein the traversal is an upward traversal.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, further including:
identifying an operation unit of the operation unit graph that is fused into the consolidated operation units block but has a dataflow to another operation unit of the operation unit graph which is outside the consolidated operation units block;
duplicating the identified operation unit and its dataflows and duplicating any other operation unit in the consolidated operation units block that provides input to the identified operation unit and its dataflows; and
based on the operation unit graph with the consolidated operation units block and the duplicated operation units and dataflows, performing the allocating and the executing.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein the architectural hints are expressed as lists of nodes and edges that translate into a pattern graph.

### Claim 7

A non-transitory computer readable storage medium impressed with computer program instructions to efficiently execute an operation unit graph on a reconfigurable data processor with a target architecture, the instructions, when executed on a processor, implement a method comprising:
reducing a number of physical compute units and/or physical memory units of the reconfigurable data processor required to execute the operation unit graph by
receiving, from a user, architectural hints that are specific to the target architecture of the reconfigurable data processor,
wherein the architectural hints
call for fusing first operation units when executing patterns of the first operation units on the physical compute units and/or physical memory units of the reconfigurable data processor,
specify the first operation units in a pattern as first nodes,
specify first dataflows among the first operation units in the pattern as first edges, and
direct fusion among the first operation units in the pattern;
scanning the operation unit graph to detect instances of the patterns of the first operation units specified by the architectural hints, including
matching second nodes and second edges in the operation unit graph with the first nodes and the first edges in the architectural hints, and detecting pattern matches;
fusing operation units of the second nodes and the second edges in the operation unit graph into a consolidated operation units block, thereby producing a fused operation unit graph;
allocating the physical compute units and/or physical memory units of the reconfigurable data processor to the fused operation unit graph; and
executing the fused operation unit graph on the reconfigurable data processor based on the allocation.

### Claim 8 (depends on 7)

The non-transitory computer readable storage medium of claim 7, wherein the architectural hints specify a first output operation unit in the pattern as a first output node.

### Claim 9 (depends on 8)

The non-transitory computer readable storage medium of claim 8, implementing the method further comprising:
detecting the pattern matches by
matching the first output node specified by the architectural hints with a second output node in the operation unit graph, and
beginning with the second output node in the operation unit graph,
traversing the operation unit graph to determine that the second nodes and the second edges in the operation unit graph match the first nodes and the first edges in the architectural hints.

### Claim 10 (depends on 9)

The non-transitory computer readable storage medium of claim 9, wherein the traversal is an upward traversal.

### Claim 11 (depends on 7)

The non-transitory computer readable storage medium of claim 7, implementing the method further comprising:
identifying an operation unit of the operation unit graph that is fused into the consolidated operation units block but has a dataflow to another operation unit of the operation unit graph which is outside the consolidated operation units block;
duplicating the identified operation unit and its dataflows and duplicating any other operation unit in the consolidated operation units block that provides input to the identified operation unit and its dataflows; and
based on the operation unit graph with the consolidated operation units block and the duplicated operation units and dataflows, performing the allocating and the executing.

### Claim 12 (depends on 7)

The non-transitory computer readable storage medium of claim 7, wherein the architectural hints are expressed as lists of nodes and edges that translate into a pattern graph.

### Claim 13

A system including one or more processors coupled to memory, the memory loaded with computer instructions to efficiently execute an operation unit graph on a reconfigurable data processor with a target architecture, the instructions, when executed on the processors, implement actions comprising:
reducing a number of physical compute units and/or physical memory units of the reconfigurable data processor required to execute the operation unit graph by
receiving, from a user, architectural hints that are specific to the target architecture of the reconfigurable data processor,
wherein the architectural hints
call for fusing first operation units when executing patterns of the first operation units on the physical compute units and/or physical memory units of the reconfigurable data processor,
specify the first operation units in a pattern as first nodes,
specify first dataflows among the first operation units in the pattern as first edges, and
direct fusion among the first operation units in the pattern;
scanning the operation unit graph to detect instances of the patterns of the first operation units specified by the architectural hints, including
matching second nodes and second edges in the operation unit graph with the first nodes and the first edges in the architectural hints, and detecting pattern matches;
fusing operation units of the second nodes and the second edges in the operation unit graph into a consolidated operation units block, thereby producing a fused operation unit graph;
allocating the physical compute units and/or physical memory units of the reconfigurable data processor to the fused operation unit graph; and
executing the fused operation unit graph on the reconfigurable data processor based on the allocation.

### Claim 14 (depends on 13)

The system of claim 13, wherein the architectural hints specify a first output operation unit in the pattern as a first output node.

### Claim 15 (depends on 14)

The system of claim 14, implementing actions further comprising:
detecting the pattern matches by
matching the first output node specified by the architectural hints with a second output node in the operation unit graph, and
beginning with the second output node in the operation unit graph,
traversing the operation unit graph to determine that the second nodes and the second edges in the operation unit graph match the first nodes and the first edges in the architectural hints.

### Claim 16 (depends on 15)

The system of claim 15, wherein the traversal is an upward traversal.

### Claim 17 (depends on 13)

The system of claim 13, implementing actions further comprising:
identifying an operation unit of the operation unit graph that is fused into the consolidated operation units block but has a dataflow to another operation unit of the operation unit graph which is outside the consolidated operation units block;
duplicating the identified operation unit and its dataflows and duplicating any other operation unit in the consolidated operation units block that provides input to the identified operation unit and its dataflows; and
based on the operation unit graph with the consolidated operation units block and the duplicated operation units and dataflows, performing the allocating and the executing.

### Claim 18 (depends on 13)

The system of claim 13, wherein the architectural hints are expressed as lists of nodes and edges that translate into a pattern graph.
