# US 2023/0053537 A1 · Multiple overlays for use with a data processing array

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0053537 A1
- Document ID: US20230053537A1
- Status: Published application
- Kind: A1
- Application number: 17/819,879
- Filed: 2022-08-15
- Published: 2023-02-23
- Earliest claimed: 2021-08-20
- Assignees: Xilinx, Inc.
- Applicants: Xilinx, Inc.
- Inventors: Baris Ozgul; David Clarke; Peter McColgan; Stephan Munz; Dylan Stuart; Pedro Miguel Parola Duarte; Juan J. Noguera Serra
- Web page: https://patentgrep.com/patent/US20230053537A1

## CPC

- G06F 9/44505: {Configuring for program initiating, e.g. using registry, configuration files}
- G06F 15/17: using an input/output type connection, e.g. channel, I/O port
- G06F 9/5083: {Techniques for rebalancing the load in a distributed system}
- G06F 13/1673: {using buffers}
- G06F 13/28: using burst mode transfer, e.g. direct memory access {DMA}, cycle steal (G06F13/32 takes precedence)
- G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)}
- G06N 3/063: using electronic means
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Using multiple overlays with a data processing array includes loading an application in a data processing array. The data processing array includes a plurality of compute tiles each having a processor. The application specifies kernels executable by the processors and implements stream channels that convey data to the plurality of compute tiles. During runtime of the application, a plurality of overlays are sequentially implemented in the data processing array. Each overlay implements a different mode of data movement in the data processing array via the stream channels. For each overlay implemented, a workload is performed by moving data to the plurality of compute tiles based on the respective mode of data movement.

## Claims

### Claim 1

A method, comprising:
loading an application in a data processing array;
wherein the data processing array includes a plurality of compute tiles each having a processor;
wherein the application specifies kernels executable by the processors and implements stream channels that convey data to the plurality of compute tiles;
during runtime of the application, sequentially implementing a plurality of overlays in the data processing array, wherein each overlay implements a different mode of data movement in the data processing array via the stream channels; and
for each overlay implemented, performing a workload by moving data to the plurality of compute tiles based on the respective mode of data movement.

### Claim 2 (depends on 1)

The method of claim 1, wherein the plurality of overlays are implemented in the data processing array for the application without loading a different application into the data processing array that loads different kernels into the compute tiles or modifies the stream channels.

### Claim 3 (depends on 1)

The method of claim 1, wherein the data processing array is subdivided into a plurality of partitions each including a subset of the plurality of compute tiles, wherein each partition is adapted to concurrently implement a different application and sequentially implement a plurality of different overlays specific to the application executed by the partition.

### Claim 4 (depends on 1)

The method of claim 1, wherein sequentially implementing a plurality of overlays comprises:
configurating the data processing array with a first overlay of the plurality of overlays to perform a first workload including a first matrix multiply operation; and
configurating the data processing array with a second overlay of the plurality of overlays to perform a second workload including a second matrix multiply operation;
wherein the first matrix multiply operation and the second matrix multiply operation are of different dimensions.

### Claim 5 (depends on 1)

The method of claim 1, wherein the application implements a neural-network and each layer of the neural-network is mapped to one of the plurality of overlays, and wherein different ones of the plurality of overlays are loaded over time to implement respective layers of the neural-network.

### Claim 6 (depends on 1)

The method of claim 1, wherein each overlay specifies a different mapping of buffers to stream channels.

### Claim 7 (depends on 1)

The method of claim 1, wherein the mode of data movement of each overlay is characterized by a number of feature maps and a number of weights conveyed over the stream channels.

### Claim 8 (depends on 1)

The method of claim 1, wherein sequentially implementing a plurality of overlays comprises:
for each overlay, programming a plurality of direct memory access circuits with a different mapping of buffers to the stream channels.

### Claim 9 (depends on 1)

The method of claim 1, further comprising:
for a selected overlay of the plurality of overlays, providing a runtime parameter to a selected compute tile of the plurality of compute tiles, wherein the runtime parameter configures an operational parameter of a kernel executed by the selected compute tile.

### Claim 10 (depends on 9)

The method of claim 9, wherein the selected overlay corresponds to a particular layer of the application, and wherein the runtime parameter specifies at least one dimension of the particular layer implemented by the selected overlay.

### Claim 11 (depends on 9)

The method of claim 9, wherein the runtime parameter selectively enables a function of the kernel executed by the selected compute tile.

### Claim 12 (depends on 1)

The method of claim 1, further comprising:
for a selected overlay of the plurality of overlays, providing a runtime parameter to a selected compute tile of the plurality of compute tiles, wherein the runtime parameter selects a kernel from a plurality of kernels of the selected compute tile for execution.

### Claim 13

A system, comprising:
a data processing array disposed in an integrated circuit, wherein the data processing array includes a plurality of compute tiles each having a processor; and
wherein the data processing array is configured to implement an application, wherein the application specifies kernels executable by the processors and stream channels that convey data to the plurality of compute tiles; and
wherein, during runtime of the application, the data processing array is adapted to implement a plurality of different overlays, wherein each overlay implements a different mode of data movement in the data processing array via the stream channels to perform a workload.

### Claim 14 (depends on 13)

The system of claim 13, wherein the application implements a neural-network and each layer of the neural-network is mapped to one of the plurality of overlays, and wherein different ones of the plurality of overlays are loaded over time to implement respective layers of the neural-network.

### Claim 15 (depends on 13)

The system of claim 13, wherein each overlay specifies a different mapping of buffers to stream channels.

### Claim 16 (depends on 13)

The system of claim 13, wherein the mode of each overlay is characterized by a number of feature maps and a number of weights conveyed over the stream channels.

### Claim 17 (depends on 13)

The system of claim 13, wherein, for a selected overlay of the plurality of overlays, a runtime parameter provided to a selected compute tile of the plurality of compute tiles configures an operational parameter of a kernel executed by the selected compute tile.

### Claim 18 (depends on 17)

The system of claim 17, wherein the selected overlay corresponds to a particular layer of the application, and wherein:
the runtime parameter specifies at least one dimension of the particular layer implemented by the selected overlay; or
the runtime parameter selectively enables a function of the kernel executed by the selected compute tile.

### Claim 19 (depends on 13)

The system of claim 13, wherein the plurality of overlays are implemented in the data processing array for the application without loading a different application into the data processing array that loads different kernels into the compute tiles or modifies the stream channels.

### Claim 20 (depends on 13)

The system of claim 13, wherein:
a first overlay of the plurality of overlays configures the data processing array to perform a first workload including a first matrix multiply operation; and
a second overlay of the plurality of overlays configures the data processing array to perform a second workload including a second matrix multiply operation;
wherein the first matrix multiply operation and the second matrix multiply operation are of different dimensions.
