# US 12,602,349 B2 · Handling dynamic tensor lengths in a reconfigurable processor that includes multiple memory units

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,602,349 B2
- Document ID: US12602349B2
- Status: Granted
- Kind: B2
- Application number: 18/213,598
- Filed: 2023-06-23
- Published: 2024-12-26
- Granted: 2026-04-14
- Earliest claimed: 2023-06-23
- Assignees: SambaNova Systems, Inc.
- Applicants: SambaNova Systems, Inc.
- Inventors: Abhishek Srivastava; Matthew Vilim; Raghu Prabhakar; Sankar Rachuru; Zhekun Zhang; Matheen Musaddiq; Apurv Vivek; Sitanshu Gupta
- Web page: https://patentgrep.com/patent/US12602349B2

## CPC

- G06F 8/41: Compilation
- G06F 15/7867: {with reconfigurable architecture}
- G06F 15/825: {Dataflow computers}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

In some aspects, a program is executed on a coarse-grained reconfigurable (CGR) processor. The CGR determines that the program produces an output that includes a variable length tensor, determines a maximum size of the variable length tensor and sets, based on the maximum size, a maximum of a counter associated with the program. The counter is set to an initial value of zero. The CGR initiates execution of the program, causing the program to receive an input tensor. Based on determining that the program is operating on a first portion of the input tensor, the CGR performs an update to the counter, to create an updated counter, and communicates the updated counter to one or more consumers within the program. After determining that the program has completed operating on the input tensor, a final size of the output is communicated to one or more downstream consumers external to the program.

## Claims

### Claim 1

A computer-implemented method to execute a program on a coarse-grained reconfigurable (CGR) processor comprising an array of CGR units, the computer-implemented method comprising:
determining that the program produces an output that includes a variable length tensor;
determining a maximum size of the variable length tensor;
setting, based on the maximum size, a maximum value of a counter associated with the program;
setting a counter value of the counter to an initial value of zero;
initiating execution of the program, wherein execution of the program causes the program to receive an input tensor;
determining that the program is operating on a first portion of the input tensor;
performing an update to the counter value of the counter, while the program is executing and based on the program operating on the first portion of the input tensor, to create an updated counter value; and
communicating the updated counter value to one or more consumers of the counter within the program.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein:
determining that the program produces the output that includes a variable length tensor is performed at a compile time.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein:
determining the maximum size of the variable length tensor is performed at a compile time.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, further comprising:
determining that the program is operating on a second portion of the input tensor;
performing a second update to the counter value of the counter, while the program is executing and based on the program operating on the second portion of the input tensor, to create a second updated counter value; and
communicating the second updated counter value to the one or more consumers of the counter within the program.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, further comprising:
determining that the program has completed operating on the input tensor;
determining a final size of the output; and
communicating the final size to one or more downstream consumers of the output.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein the counter is located in:
a read context of a memory unit that is used when reading the first portion of the input tensor; or
a write context of the memory unit that is used when writing a portion of the output.

### Claim 7 (depends on 6)

The computer-implemented method of claim 6, wherein:
the write context performs a write operation to a first buffer in a scratchpad of the memory unit;
the read context performs a read operation from a second buffer in the scratchpad of the memory unit; and
the read operation and the write operation overlap in time.

### Claim 8

A non-transitory computer-readable storage medium storing computer program instructions that, when executed on a processor, perform operations comprising:
determining that a program produces an output that includes a variable length tensor;
determining a maximum size of the variable length tensor;
setting, based on the maximum size, a maximum value of a counter associated with the program;
setting a counter value of the counter to an initial value of zero;
initiating execution of the program, wherein execution of the program causes the program to receive an input tensor;
determining that the program is operating on a first portion of the input tensor;
performing an update to the counter value of the counter, while the program is executing and based on the program operating on the first portion of the input tensor, to create an updated counter value; and
communicating the updated counter value to one or more consumers of the counter within the program.

### Claim 9 (depends on 8)

The non-transitory computer-readable storage medium of claim 8, wherein:
determining that the program produces the output that includes a variable length tensor is performed at compile time.

### Claim 10 (depends on 8)

The non-transitory computer-readable storage medium of claim 8, wherein:
determining the maximum size of the variable length tensor is performed at compile time.

### Claim 11 (depends on 8)

The non-transitory computer-readable storage medium of claim 8, the operations further comprising:
determining that the program is operating on a second portion of the input tensor;
performing a second update to the counter value of the counter, while the program is executing and based on the program operating on the second portion of the input tensor, to create a second updated counter value; and
communicating the second updated counter value to the one or more consumers of the counter within the program.

### Claim 12 (depends on 8)

The non-transitory computer-readable storage medium of claim 8, the operations further comprising:
determining that the program has completed operating on the input tensor;
determining a final size of the outputs; and
communicating the final size to one or more downstream consumers of the output of the program.

### Claim 13 (depends on 8)

The non-transitory computer-readable storage medium of claim 8, wherein the counter is located in:
a read context of a memory unit that is used when reading the first portion of the input tensor; or
a write context of the memory unit that is used when writing a portion of the output.

### Claim 14 (depends on 13)

The non-transitory computer-readable storage medium of claim 13, wherein:
the write context performs a write operation to a first buffer in a scratchpad of the memory unit;
the read context performs a read operation from a second buffer in the scratchpad of the memory unit; and
the read operation and the write operation overlap in time.

### Claim 15

A system comprising one or more processors coupled to a memory device, the memory device to store computer program instructions that are executable by the one or more processors to perform operations comprising:
determining that a program produces an output that includes a variable length tensor;
determining a maximum size of the variable length tensor;
setting, based on the maximum size, a maximum value of a counter associated with the program;
setting a counter value of the counter to an initial value of zero;
initiating execution of the program, wherein execution of the program causes the program to receive an input tensor;
determining that the program is operating on a first portion of the input tensor;
performing an update to the counter value of the counter, while the program is executing and based on the program operating on the first portion of the input tensor, to create an updated counter value; and
communicating the updated counter value to one or more consumers of the counter within the program.

### Claim 16 (depends on 15)

The system of claim 15, wherein:
determining that the program produces the output that includes a variable length tensor is performed at a compile time.

### Claim 17 (depends on 15)

The system of claim 15, wherein:
determining the maximum size of the variable length tensor is performed at a compile time.

### Claim 18 (depends on 15)

The system of claim 15, the operations further comprising:
determining that the program is operating on a second portion of the input tensor;
performing a second update to the counter value of the counter, while the program is executing and based on the program operating on the second portion of the input tensor, to create a second updated counter value; and
communicating the second updated counter value to the one or more consumers of the counter within the program.

### Claim 19 (depends on 15)

The system of claim 15, the operations further comprising:
determining that the program has completed operating on the input tensor;
determining a final size of the output; and
communicating the final size to one or more downstream consumers of the output of the program.

### Claim 20 (depends on 15)

The system of claim 15, wherein:
the counter is located in at least one of:
a read context of a memory unit that is used when reading the first portion of the input tensor; or
a write context of the memory unit that is used when writing a portion of the output;
the write context performs a write operation to a first buffer in a scratchpad of the memory unit;
the read context performs a read operation from a second buffer in the scratchpad of the memory unit; and
the read operation and the write operation overlap in time.