# US 2024/0394205 A1 · Intelligence processing unit and its tensor concatenation method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0394205 A1
- Document ID: US20240394205A1
- Status: Published application
- Kind: A1
- Application number: 18/427,853
- Filed: 2024-01-31
- Published: 2024-11-28
- Earliest claimed: 2023-05-25
- Assignees: Sigmastar Technology Ltd.
- Applicants: SigmaStar Technology Ltd.
- Inventors: Hu He; Jian Liu
- Web page: https://patentgrep.com/patent/US20240394205A1

## CPC

- G06F 13/28: using burst mode transfer, e.g. direct memory access {DMA}, cycle steal (G06F13/32 takes precedence)
- G06F 2213/28: DMA
- Y02D 10/00: Energy efficient computing, e.g. low power processors, power management or thermal management
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE

## Abstract

An intelligence processing unit is coupled to an external memory and includes a memory, a direct memory access (DMA) circuit, and a vector accelerator. The external memory stores a first tensor and a second tensor. The DMA circuit performs the following steps: reading a first part of the first tensor from the external memory; storing the first part of the first tensor in the memory; reading a second part of the second tensor from the external memory; and storing the second part of the second tensor in the memory. The vector accelerator includes a register circuit and performs the following steps: storing P bytes of the first part of the first tensor in a target row of the register circuit; storing Q bytes of the second part of the second tensor in the target row of the register circuit; and writing data of the target row into the memory.

## Claims

### Claim 1

An intelligence processing unit (IPU) coupled to an external memory storing a first tensor and a second tensor, the IPU comprising:
a memory;
a direct memory access (DMA) circuit coupled to the external memory and the memory and configured to perform following steps:
reading a first part of the first tensor from the external memory;
storing the first part of the first tensor in the memory;
reading a second part of the second tensor from the external memory; and
storing the second part of the second tensor in the memory; and
a vector accelerator that comprises a register circuit, is coupled to the memory, and is configured to perform following steps:
storing P bytes of the first part of the first tensor in a target row of the register circuit, P being a positive integer;
storing Q bytes of the second part of the second tensor in the target row of the register circuit, Q being a positive integer; and
writing data of the target row into the memory.

### Claim 2 (depends on 1)

The IPU of claim 1, wherein the target row is a first target row, and the vector accelerator is further configured to perform following steps:
storing R bytes of the first part of the first tensor in a second target row of the register circuit, R being a positive integer;
wherein the second target row is different from the first target row, and P is equal to R.

### Claim 3 (depends on 2)

The IPU of claim 2, wherein the second target row is next to the first target row.

### Claim 4 (depends on 2)

The IPU of claim 2, wherein storing of the P bytes in the first target row of the register circuit and storing of the R bytes in the second target row of the register circuit are completed in a same one write operation of the vector accelerator.

### Claim 5 (depends on 2)

The IPU of claim 2, wherein the vector accelerator is further configured to perform following steps:
storing S bytes of the second part of the second tensor in the second target row of the register circuit, S being equal to Q; and
writing the first target row and the second target row into the memory simultaneously;
wherein the vector accelerator writes at most W bytes in one write operation to the memory, and W is greater than or equal to a sum of P, Q, R, and S.

### Claim 6 (depends on 1)

The IPU of claim 1, wherein the innermost dimension of the first tensor is P, and the innermost dimension of the second tensor is Q.

### Claim 7 (depends on 1)

The IPU of claim 1, wherein in the memory, a ratio of the first part of the first tensor to the second part of the second tensor is P/Q.

### Claim 8 (depends on 1)

The IPU of claim 1, wherein whenever the DMA circuit reads a part of the first tensor P times from the external memory, the DMA circuit reads a part of the second tensor Q times from the external memory.

### Claim 9 (depends on 1)

The IPU of claim 1, wherein the DMA circuit further performs following steps:
writing an effective data of the target row into the external memory;
wherein an amount of the effective data is greater than or equal to a sum of P and Q.

### Claim 10

A tensor concatenation method implemented in an intelligence processing unit (IPU), wherein the IPU is coupled to an external memory and comprises a memory and a register circuit, and the external memory stores a first tensor and a second tensor, the tensor concatenation method comprising:
reading a first part of the first tensor from the external memory;
storing the first part of the first tensor in the memory;
reading a second part of the second tensor from the external memory; and
storing the second part of the second tensor in the memory; and
storing P bytes of the first part of the first tensor in a target row of the register circuit, P being a positive integer;
storing Q bytes of the second part of the second tensor in the target row of the register circuit, Q being a positive integer; and
writing data of the target row into the memory.

### Claim 11 (depends on 10)

The tensor concatenation method of claim 10, wherein the target row is a first target row, the tensor concatenation method further comprising:
storing R bytes of the first part of the first tensor in a second target row of the register circuit, R being a positive integer;
wherein the second target row is different from the first target row, and P is equal to R.

### Claim 12 (depends on 11)

The tensor concatenation method of claim 11, wherein the second target row is next to the first target row.

### Claim 13 (depends on 11)

The tensor concatenation method of claim 11, wherein storing of the P bytes in the first target row of the register circuit and storing of the R bytes in the second target row of the register circuit are completed in a same one write operation.

### Claim 14 (depends on 11)

The tensor concatenation method of claim 11 further comprising:
storing S bytes of the second part of the second tensor in the second target row of the register circuit, S being equal to Q; and
writing the first target row and the second target row into the memory simultaneously;
wherein one write operation to the memory writes at most W bytes, and W is greater than or equal to a sum of P, Q, R, and S.

### Claim 15 (depends on 10)

The tensor concatenation method of claim 10, wherein the innermost dimension of the first tensor is P, and the innermost dimension of the second tensor is Q.

### Claim 16 (depends on 10)

The tensor concatenation method of claim 10, wherein in the memory, a ratio of the first part of the first tensor to the second part of the second tensor is P/Q.

### Claim 17 (depends on 10)

The tensor concatenation method of claim 10, wherein whenever the first tensor is read P times from the external memory, the second tensor is read Q times from the external memory.

### Claim 18 (depends on 10)

The tensor concatenation method of claim 10 further comprising:
writing an effective data of the target row into the external memory;
wherein an amount of the effective data is greater than or equal to a sum of P and Q.
